Skip to content

docs(oss): add contributor PR triage criteria and a skill that applies them - #29961

Open
wmadden-electric wants to merge 1 commit into
mainfrom
worktree/pr-evaluation-criteria-e158e8
Open

docs(oss): add contributor PR triage criteria and a skill that applies them#29961
wmadden-electric wants to merge 1 commit into
mainfrom
worktree/pr-evaluation-criteria-e158e8

Conversation

@wmadden-electric

@wmadden-electric wmadden-electric commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Linked issue

n/a — came out of triaging the current external PR queue.

Summary

Triaging incoming fork PRs had no written criteria. Each maintainer re-derived them, and an agent could not do the first pass at all. This adds the criteria as a doc and a skill that applies them.

Two facts shape the whole thing. Our CI does not run on a fork PR until a maintainer approves it, so the first decision is a safety decision — taken before we know whether the change is any good. And a contributor cannot see our roadmap, so direction-fit is ours to judge and ours to explain quickly, before they spend another week on it.

The safety criterion came out deliberately wider than "no changes to GitHub Actions". Workflow files are one way to run code on a runner, not the only one. A new script that an existing workflow step invokes, an install hook, a build config under packages/0-config/, a test file, or a lockfile entry pointing off-registry all reach the same place. The doc lists them, and points at the fork-PR posture already documented in supply-chain.md — read-only token, no secrets, GitHub-hosted runners, no pull_request_target — which is what bounds the blast radius and makes approving fork CI routine rather than fraught.

The rest of the criteria: read the version line off the base branch (main is 8.x and takes fixes and aligned features; v7 and 7.9.x take bug fixes only); verify a claimed bug against a file and line rather than trusting the fix: prefix; separate "CI has never run" from "CI is failing", because on a fork PR those look identical and mean opposite things; judge direction against ROADMAP.md, the ADRs, and projects/; and start the staleness clock when the ball entered the contributor's court rather than at their last push, since a PR waiting on us is never stale.

One criterion came straight out of the queue: check a commenter's permission before treating their feedback as a review signal. Several open PRs carry confident review comments from accounts with read access, and contributors have changed their implementations in response. On one closed PR a comment asserted a "maintainer decision" that had never been taken, and the contributor rewrote their change on the strength of it. Permission is necessary but not sufficient — the claim has to be true too, and decisions that matter get recorded on the PR by a maintainer in their own words.

Shape

  • docs/oss/pr-triage.md — the criteria and the reasoning, ending in a fixed verdict vocabulary so triage output is comparable across PRs and across people.
  • skills-contrib/triage-contributor-pr/SKILL.md — the runnable form, with the commands for each step. It cites the doc for the criteria rather than restating them, so there is one source of truth. Read-only by default: it reports verdicts and does not close PRs, comment, or approve workflow runs.
  • docs/oss/README.md — index entry.

This mirrors how CONTRIBUTING.md and the contrib-pr skill already split policy from procedure, on the contributor side.

Testing performed

  • pnpm lint:skills — passes.
  • Applied the criteria end to end against the 12 open external PRs to check they actually decide things. They separate cleanly into merge candidates, direction calls, and one PR blocked on DCO and CLA, and they caught two things a shorter checklist would have missed: a PR that adds a CI-invoked script without changing a workflow trigger, and the non-maintainer review comments described above.

Docs and a skill only — no source or CI changes.

Summary by CodeRabbit

  • Documentation
    • Added maintainer guidance for safely triaging unsolicited external pull requests.
    • Documented checks for code safety, scope, implementation validity, permissions, project fit, and required evidence.
    • Added a read-only triage workflow with standardized verdicts and contributor response guidance.
    • Linked the new guidance from the open-source documentation and related triage resources.

…s them

Triaging incoming fork PRs had no written criteria, so each maintainer
re-derived them and an agent could not do the first pass at all.

Two things drive the shape. Our CI does not run on a fork PR until a
maintainer approves it, so the first decision is a safety decision taken
before we know whether the change is any good. And a contributor cannot
see our roadmap, so direction-fit is ours to judge and ours to explain
quickly.

The safety criterion is deliberately wider than "no workflow changes".
Workflow files are one way to run code on a runner, not the only one: a
new script an existing step invokes, an install hook, a build config, a
test, or a lockfile entry pointing off-registry all reach the same place.
The doc lists them and points at the fork-PR posture in supply-chain.md
that bounds the blast radius.

The rest covers reading the version line off the base branch, verifying a
claimed bug against file and line rather than the commit prefix,
separating "CI not yet run" from "CI failing", judging direction against
ROADMAP/ADRs/projects, and starting the staleness clock when the ball
entered the contributor's court rather than at their last push.

One criterion came out of the queue itself: check a commenter's
permission before treating their feedback as a review signal.
Contributors act on confident review comments, and we have already seen a
claimed "maintainer decision" that was never taken change a contribution.

Signed-off-by: willbot <w.a.madden+machine@gmail.com>
Signed-off-by: Will Madden <madden@prisma.io>
@wmadden-electric
wmadden-electric requested a review from a team as a code owner August 11, 2026 07:58
@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Added OSS documentation and a read-only skill for safely triaging unsolicited external contributor pull requests. The guidance covers safety, scope, validation, permissions, staleness, direction fit, and standardized verdicts.

Changes

External PR triage

Layer / File(s) Summary
Triage guidance and verdicts
docs/oss/pr-triage.md
Defines safety checks, version and scope validation, mechanical requirements, direction fit, staleness rules, permission checks, and evidence-based verdicts.
Read-only triage skill
skills-contrib/triage-contributor-pr/SKILL.md
Adds commands and procedures for collecting PR data, scanning changes, validating claims, checking permissions and status, and reporting results without performing write actions.
Documentation navigation
docs/oss/README.md
Links the triage guide from the OSS audience map and directory index.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested labels: lgtm

Suggested reviewers: wmadden

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the added contributor PR triage criteria, companion skill, and related documentation changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch worktree/pr-evaluation-criteria-e158e8

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pkg-pr-new

pkg-pr-new Bot commented Aug 11, 2026

Copy link
Copy Markdown

Open in StackBlitz

prisma-next

npm i https://pkg.pr.new/prisma-next@29961

@prisma/orm-extension-arktype-json

npm i https://pkg.pr.new/@prisma/orm-extension-arktype-json@29961

@prisma/orm-extension-middleware-cache

npm i https://pkg.pr.new/@prisma/orm-extension-middleware-cache@29961

@prisma/orm-extension-paradedb

npm i https://pkg.pr.new/@prisma/orm-extension-paradedb@29961

@prisma/orm-extension-pgvector

npm i https://pkg.pr.new/@prisma/orm-extension-pgvector@29961

@prisma/orm-extension-postgis

npm i https://pkg.pr.new/@prisma/orm-extension-postgis@29961

@prisma/orm-extension-supabase

npm i https://pkg.pr.new/@prisma/orm-extension-supabase@29961

@prisma/orm-family-mongo

npm i https://pkg.pr.new/@prisma/orm-family-mongo@29961

@prisma/orm-family-sql

npm i https://pkg.pr.new/@prisma/orm-family-sql@29961

@prisma/orm-framework

npm i https://pkg.pr.new/@prisma/orm-framework@29961

@prisma/orm-mongo

npm i https://pkg.pr.new/@prisma/orm-mongo@29961

@prisma/orm-postgres

npm i https://pkg.pr.new/@prisma/orm-postgres@29961

@prisma/orm-sqlite

npm i https://pkg.pr.new/@prisma/orm-sqlite@29961

@prisma/orm-target-mongo

npm i https://pkg.pr.new/@prisma/orm-target-mongo@29961

@prisma/orm-target-postgres

npm i https://pkg.pr.new/@prisma/orm-target-postgres@29961

@prisma/orm-target-sqlite

npm i https://pkg.pr.new/@prisma/orm-target-sqlite@29961

@prisma/orm-toolchain

npm i https://pkg.pr.new/@prisma/orm-toolchain@29961

commit: 93578ba

@github-actions

Copy link
Copy Markdown
Contributor

size-limit report 📦

Path Size
postgres / no-emit 169.92 KB (0%)
postgres / emit 147.6 KB (0%)
mongo / no-emit 100.42 KB (0%)
mongo / emit 90.25 KB (0%)
cf-worker / no-emit 194.11 KB (0%)
cf-worker / emit 169.49 KB (0%)

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@skills-contrib/triage-contributor-pr/SKILL.md`:
- Around line 89-91: Update the “6. Mechanics” section to include checks for a
conventional commit title, one logical change, and tests updated in the same PR,
alongside the existing sign-off, CLA, and CI checks. Ensure the skill evaluates
and reports all documented mechanical requirements before giving a positive
verdict.
- Around line 31-34: Update the pull-request listing command in the documented
workflow so the contributor filter is supplied as an input rather than hardcoded
to “snowingfox” and “wehamed”, and remove the 300-result truncation by
enumerating or paginating through the complete open-PR queue. Preserve the
existing output fields and formatting while ensuring all requested matching
contributors are included.
- Around line 81-87: Update the per-PR bug-verification procedure around the
issue lookup and source confirmation to require explicit pass/fail evidence for
all gates: verify the linked issue is open, reproduce the bug against the base
branch, and demonstrate a test fails without the proposed change. Require the
final bug-related verdict to report each check’s result, while preserving the
existing source-layer confirmation and plain disclosure when reproduction is
unavailable.
- Around line 68-71: Expand the CI security scan around the workflow inspection
commands to cover the complete .github execution surface, including all workflow
files, composite actions, and other runner-side configuration rather than only
.github/workflows/ci.yml. Ensure the checks detect pull_request_target, secrets
usage, runs-on, and related unsafe settings across every applicable file under
.github/.
- Line 94: Update the jq expression for each commit to validate that the
Signed-off-by trailer identity matches the commit author, rather than checking
trailer presence alone. For GitHub web-UI merge commits, use the DCO app status
as the source of truth instead of the trailer comparison.
- Line 96: Update the triage flow to reuse the saved PR snapshot’s
statusCheckRollup from Step 2 instead of issuing a separate gh api check-runs
request. Derive the reported check statuses from both CheckRun and StatusContext
entries in statusCheckRollup, preserving legacy status checks and snapshot
consistency.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: a7c3f848-1b7c-478b-8717-6f87ec91b572

📥 Commits

Reviewing files that changed from the base of the PR and between 7e80e77 and 93578ba.

📒 Files selected for processing (3)
  • docs/oss/README.md
  • docs/oss/pr-triage.md
  • skills-contrib/triage-contributor-pr/SKILL.md

Comment on lines +31 to +34
gh pr list --repo prisma/prisma --state open --limit 300 \
--json number,title,author,baseRefName,createdAt,updatedAt,isDraft \
--jq '.[] | select([.author.login | ascii_downcase] | inside(["snowingfox","wehamed"])) | "\(.number)\t\(.author.login)\t\(.baseRefName)\t\(.updatedAt)\t\(.title)"'
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Make the contributor filter an input.

This command silently selects only snowingfox and wehamed. It also stops at 300 PRs. The documented full-queue workflow can therefore omit valid external PRs.

Accept the requested author list as input, or enumerate the complete queue. Paginate until all matching PRs are returned.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` around lines 31 - 34, Update
the pull-request listing command in the documented workflow so the contributor
filter is supplied as an input rather than hardcoded to “snowingfox” and
“wehamed”, and remove the 300-result truncation by enumerating or paginating
through the complete open-PR queue. Preserve the existing output fields and
formatting while ensuring all requested matching contributors are included.

Comment on lines +68 to +71
```bash
grep -n "^on:" -A4 .github/workflows/ci.yml
grep -rn "pull_request_target\|secrets\.\|runs-on" .github/workflows/ci.yml
```

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

Scan the complete CI execution surface.

The procedure checks only .github/workflows/ci.yml. The criteria require inspection of all workflows, composite actions, and other runner-side files under .github/. A second workflow can contain pull_request_target, secrets, or unsafe permissions without this check finding it.

Proposed scan scope
-grep -n "^on:" -A4 .github/workflows/ci.yml
-grep -rn "pull_request_target\|secrets\.\|runs-on" .github/workflows/ci.yml
+rg -n '^(on:|.*pull_request_target|.*secrets\.|.*permissions:|.*runs-on)' .github/
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
```bash
grep -n "^on:" -A4 .github/workflows/ci.yml
grep -rn "pull_request_target\|secrets\.\|runs-on" .github/workflows/ci.yml
```
🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` around lines 68 - 71, Expand
the CI security scan around the workflow inspection commands to cover the
complete .github execution surface, including all workflow files, composite
actions, and other runner-side configuration rather than only
.github/workflows/ci.yml. Ensure the checks detect pull_request_target, secrets
usage, runs-on, and related unsafe settings across every applicable file under
.github/.

Comment on lines +81 to +87
For each PR, confirm the linked issue is real and matches:

```bash
gh issue view <n> --repo prisma/prisma --json title,state,author,createdAt
```

Then confirm the bug in the checked-out source — find the line that ignores the input, or the `TODO` that parks it — and confirm the fix reaches the layer that actually has the bug. Cite `file:line`. If you could not reproduce something, say so plainly instead of implying you did.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Require every documented bug-verification gate.

The procedure does not explicitly require the issue to be open, a reproduction against the base branch, or a test that fails without the change. Add explicit pass/fail evidence for each check before returning a bug-related verdict.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` around lines 81 - 87, Update
the per-PR bug-verification procedure around the issue lookup and source
confirmation to require explicit pass/fail evidence for all gates: verify the
linked issue is open, reproduce the bug against the base branch, and demonstrate
a test fails without the proposed change. Require the final bug-related verdict
to report each check’s result, while preserving the existing source-layer
confirmation and plain disclosure when reproduction is unavailable.

Comment on lines +89 to +91
### 6. Mechanics

Sign-off per commit, CLA status, and whether CI has ever run:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Complete the mechanical requirements.

The documentation also requires a conventional commit title, one logical change, and tests updated in the same PR. This step checks only sign-off, CLA, and CI, so the skill can report a positive verdict while missing required checks.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` around lines 89 - 91, Update
the “6. Mechanics” section to include checks for a conventional commit title,
one logical change, and tests updated in the same PR, alongside the existing
sign-off, CLA, and CI checks. Ensure the skill evaluates and reports all
documented mechanical requirements before giving a positive verdict.

Sign-off per commit, CLA status, and whether CI has ever run:

```bash
jq -r '.commits[] | "\(.oid[0:8]) signoff=\(if (.messageBody // "") | test("Signed-off-by:") then "YES" else "NO" end)"' wip/pr-triage/pr-<n>.json

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔒 Security & Privacy | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- target file ---'
sed -n '70,110p' skills-contrib/triage-contributor-pr/SKILL.md
printf '%s\n' '--- triage documentation references ---'
rg -n -C 4 -i 'DCO|sign.?off|merge commit|mechanical|trailer|author' docs/oss/pr-triage.md skills-contrib/triage-contributor-pr/SKILL.md
printf '%s\n' '--- relevant file outline ---'
ast-grep outline skills-contrib/triage-contributor-pr/SKILL.md --match jq --view expanded

Repository: prisma/prisma

Length of output: 13583


🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- target file ---'
sed -n '70,110p' skills-contrib/triage-contributor-pr/SKILL.md
printf '%s\n' '--- documentation ---'
rg -n -C 5 -i 'DCO|sign.?off|merge commit|mechanical|trailer|author' docs/oss/pr-triage.md
printf '%s\n' '--- skill references ---'
rg -n -C 3 -i 'DCO|sign.?off|merge commit|mechanical|trailer|author|commits' skills-contrib/triage-contributor-pr/SKILL.md

Repository: prisma/prisma

Length of output: 10484


🏁 Script executed:

#!/bin/bash
set -eu
printf '%s\n' '--- DCO policy ---'
rg -n -C 8 -i 'Developer Certificate|DCO|Signed-off-by|author' CONTRIBUTING.md
printf '%s\n' '--- commit fixture/data files ---'
git ls-files | rg '(^|/)(pr-[^/]+\.json|.*triage.*\.json|.*fixture.*)$' | head -80
printf '%s\n' '--- jq behavior: presence versus identity ---'
python3 - <<'PY'
import json, subprocess, textwrap
samples = [
    ("matching", "Fix bug\n\nSigned-off-by: Alice Example <alice@example.com>", "Alice Example <alice@example.com>"),
    ("mismatching", "Fix bug\n\nSigned-off-by: Bob Example <bob@example.com>", "Alice Example <alice@example.com>"),
    ("absent", "Fix bug\n", "Alice Example <alice@example.com>"),
]
expr = r'''if (.messageBody // "") | test("Signed-off-by:") then "YES" else "NO" end'''
for name, body, author in samples:
    obj = {"messageBody": body, "author": {"name": author.split(" <", 1)[0], "email": author.split("<", 1)[1][:-1]}}
    out = subprocess.check_output(["jq", "-r", expr], input=json.dumps(obj).encode()).decode().strip()
    print(f"{name}: author={author!r}, expression={out}")
PY

Repository: prisma/prisma

Length of output: 13522


Validate DCO identity against the commit author.

This expression checks only for trailer presence. It accepts an unrelated Signed-off-by: identity. Compare the trailer with the commit author, and use the DCO app status for GitHub web-UI merge commits.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` at line 94, Update the jq
expression for each commit to validate that the Signed-off-by trailer identity
matches the commit author, rather than checking trailer presence alone. For
GitHub web-UI merge commits, use the DCO app status as the source of truth
instead of the trailer comparison.

```bash
jq -r '.commits[] | "\(.oid[0:8]) signoff=\(if (.messageBody // "") | test("Signed-off-by:") then "YES" else "NO" end)"' wip/pr-triage/pr-<n>.json
jq -r '.comments[] | select(.author.login=="CLAassistant") | .body[0:200]' wip/pr-triage/pr-<n>.json
gh api "repos/prisma/prisma/commits/<head-sha>/check-runs" --jq '[.check_runs[] | "\(.name)=\(.conclusion // .status)"] | join(" ")'

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -eu

file='skills-contrib/triage-contributor-pr/SKILL.md'
printf '%s\n' '--- target file: relevant sections ---'
sed -n '1,125p' "$file"

printf '%s\n' '--- references to statusCheckRollup, check-runs, and fetch instructions ---'
rg -n -C 3 'statusCheckRollup|check-runs|fetch|once|wip/pr-triage' "$file" docs/oss/pr-triage.md 2>/dev/null || true

printf '%s\n' '--- saved snapshot examples and schema references ---'
rg -n -C 3 'pr-[0-9]+\.json|statusCheckRollup|checkRuns|check-runs' . --glob '!node_modules' --glob '!dist' --glob '!build' 2>/dev/null | head -300

Repository: prisma/prisma

Length of output: 15322


🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
from pathlib import Path
import re

text = Path("skills-contrib/triage-contributor-pr/SKILL.md").read_text()
fetch = re.search(r'--json\s+([^\\\n]+)', text)
assert fetch, "Step 2 JSON selection was not found"
fields = [f.strip() for f in fetch.group(1).split(",")]
assert "statusCheckRollup" in fields
assert "headRefOid" not in fields
assert "headRefName" not in fields
assert "gh api .*check-runs" in text
print("Step 2 fields:", fields)
print("statusCheckRollup is saved:", "statusCheckRollup" in fields)
print("Separate check-runs API command present:", bool(re.search(r'gh api .*check-runs', text)))
print("Head SHA is not saved by the listed gh pr view fields:", "headRefOid" not in fields)
PY

Repository: prisma/prisma

Length of output: 239


🌐 Web query:

GitHub CLI gh pr view --json statusCheckRollup field CheckRun StatusContext documentation

💡 Result:

The statusCheckRollup field in the GitHub CLI (gh pr view --json statusCheckRollup) returns a collection of status checks for a pull request [1][2]. This field provides a union of two distinct object types representing different check systems: CheckRun (GitHub Actions) and StatusContext (legacy commit status checks like those from third-party CI services) [3][4]. Because these two types originate from different systems, their data structures and fields for reporting status differ [4]. When working with the output of statusCheckRollup, you should use the appropriate field based on the object type [5]: 1. CheckRun: Represents automated check runs (e.g., GitHub Actions). These use the conclusion field to report their outcome (e.g., SUCCESS, FAILURE, CANCELLED) [6][3][4]. Their status field (e.g., IN_PROGRESS, COMPLETED) reflects the lifecycle state rather than the final verdict [4]. 2. StatusContext: Represents legacy commit status checks. These do not have a conclusion field; instead, they report their outcome using the state field (e.g., SUCCESS, PENDING, FAILURE, ERROR) [3][4]. To programmatically determine the status of a check, a common practice is to prioritize the conclusion field, falling back to the state field if conclusion is null or absent [4][5]. This pattern allows you to handle both modern and legacy check systems consistently [3][5]. For more information on available fields for the gh pr view command, you can run the command without arguments for the --json flag: gh pr view --json (which lists all queryable JSON fields) [7]. Documentation for the command is available via gh help pr view or in the official GitHub CLI manual [8].

Citations:


🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
from pathlib import Path

lines = Path("skills-contrib/triage-contributor-pr/SKILL.md").read_text().splitlines()
json_line = next(line for line in lines if "--json number,title" in line)
api_line = next(line for line in lines if "check-runs" in line)

fields = json_line.split("--json ", 1)[1].rstrip(" \\").split(",")
print("statusCheckRollup saved:", "statusCheckRollup" in fields)
print("saved field count:", len(fields))
print("separate check-runs request:", api_line.strip())
print("snapshot-only instruction:", any("Do not re-run `gh`" in line for line in lines))
PY

Repository: prisma/prisma

Length of output: 397


🌐 Web query:

site:cli.github.com/manual/gh_pr_view statusCheckRollup CheckRun StatusContext

💡 Result:

In the GitHub GraphQL API, StatusCheckRollup, CheckRun, and StatusContext are distinct components used to track and report the status of commits and pull requests [1][2]. StatusCheckRollup is an object that provides a consolidated view of the various status checks for a specific commit [2]. It acts as a wrapper or summary that includes both individual commit statuses and GitHub Actions/Check runs [2]. StatusContext represents an individual legacy commit status, often created by external integrations or tools that use the commit status API [3]. It contains details such as the state (e.g., success, failure, pending), context name, and target URL [3]. CheckRun refers to an individual execution of a check suite, which is part of the newer GitHub Checks API [1]. Unlike StatusContext, which is a simpler status report, a CheckRun typically provides more granular data, including output, annotations, and structured reports associated with a specific workflow or check [1]. These components are frequently used together when querying the state of a pull request or commit to determine if all required CI/CD processes have passed [4][2]. For example, the statusCheckRollup field on a commit allows you to fetch a summary of all these check and status types in one request [2].

Citations:


🏁 Script executed:

#!/bin/bash
set -eu

python3 - <<'PY'
from pathlib import Path

lines = Path("skills-contrib/triage-contributor-pr/SKILL.md").read_text().splitlines()
for i, line in enumerate(lines, 1):
    if "--json" in line or "statusCheckRollup" in line or "check-runs" in line:
        print(f"{i}: {line!r}")

start = next(i for i, line in enumerate(lines) if "--json" in line)
command = "\n".join(lines[start:start+3])
print("Step 2 command fragment:", repr(command))
print("statusCheckRollup occurs in Step 2 fragment:", "statusCheckRollup" in command)
print("statusCheckRollup occurrences in file:", sum("statusCheckRollup" in line for line in lines))
PY

printf '%s\n' '--- exact numbered source lines ---'
sed -n '48,56p;91,100p' skills-contrib/triage-contributor-pr/SKILL.md | cat -vet

Repository: prisma/prisma

Length of output: 2249


🌐 Web query:

GitHub CLI gh pr view --json statusCheckRollup official manual

💡 Result:

The statusCheckRollup field is an available field for the GitHub CLI command gh pr view when using the --json flag [1][2]. This field provides a summary of the CI/CD status checks associated with a pull request [3]. When running the command, you can specify this field as part of a comma-separated list of fields: gh pr view --json title,number,statusCheckRollup Note that statusCheckRollup is part of the standard list of supported JSON fields for the command [1][2]. However, it is important to be aware of the following technical considerations: 1. Permission Requirements: The field requires specific API permissions [4]. Users employing fine-grained personal access tokens (PATs) on organization-owned private repositories have reported issues where the command fails if the token lacks the necessary scopes to access status check information, as the command attempts to eagerly fetch this data [4]. 2. Data Specifics: Users have noted that the output of statusCheckRollup via gh pr view may differ in structure or content compared to data returned by the gh pr checks command [5]. 3. Troubleshooting: If you encounter errors (such as 403 Forbidden or "Resource not accessible") when using this field, you can work around the issue by explicitly requesting a subset of other fields in the --json flag that do not require those specific permissions [4].

Citations:


Reuse statusCheckRollup from the saved PR snapshot.

Step 2 already saves statusCheckRollup, which includes both CheckRun and StatusContext entries. The separate /check-runs request can return a different snapshot and omit legacy statuses. Read the saved field, or fetch the required check data during Step 2.

🧰 Tools
🪛 SkillSpector (2.5.1)

[error] 95: [YR4] YARA rule 'agent_skill_prompt_injection_hidden_instructions': Prompt injection or hidden instructions embedded in AI agent skill text [agent_skills]: YARA rule matched a hack tool or exploit indicator (offensive tools, reconnaissance, privilege escalation, or exploit frameworks).

Remediation: Remove offensive tool references and exploit code. Legitimate agent skills should not contain penetration testing tools, exploit frameworks, or reconnaissance utilities.

(YARA Match (YR4))

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@skills-contrib/triage-contributor-pr/SKILL.md` at line 96, Update the triage
flow to reuse the saved PR snapshot’s statusCheckRollup from Step 2 instead of
issuing a separate gh api check-runs request. Derive the reported check statuses
from both CheckRun and StatusContext entries in statusCheckRollup, preserving
legacy status checks and snapshot consistency.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant