feat(github-pr-reviewer): hide previous automation comments after posting new review - #466
Open
all-hands-bot wants to merge 5 commits into
Open
feat(github-pr-reviewer): hide previous automation comments after posting new review#466all-hands-bot wants to merge 5 commits into
all-hands-bot wants to merge 5 commits into
Conversation
…ting new review After posting a new review comment, the automation now hides (minimizes with the 'outdated' reason) all previous comments it posted on the PR, including the 'OpenHands is reviewing this PR' acknowledgement comment and any earlier review results. This keeps the PR conversation clean when multiple review cycles happen, since GitHub struggles to render PRs with many comments. Uses the GitHub GraphQL minimizeComment mutation with the OUTDATED classifier. Hidden comments can still be expanded by users or unhidden by moderators. Co-authored-by: openhands <openhands@all-hands.dev>
… comments The previous implementation only minimized issue comments (the 'OpenHands is reviewing this PR' acknowledgement and review results posted as comments). But the actual code reviews are PR review objects with inline diff comments — a different GitHub entity that also implements the Minimizable interface. Now _hide_previous_automation_comments handles both: 1. Issue comments — minimized as before (acknowledgement + review-as-comment) 2. PR review objects — both the review itself and each inline review comment are minimized via the same minimizeComment GraphQL mutation with OUTDATED Added _list_pr_reviews() to fetch reviews via GraphQL, and _is_automation_content() to identify automation-posted reviews by checking for the AI disclosure marker or other known signatures in the review body or inline comments. Verified against the live GitHub GraphQL schema that both PullRequestReview and PullRequestReviewComment implement the Minimizable interface. Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
Author
|
👋 This PR needs a couple of things fixed before OpenHands can review it:
Push an update once this is addressed and this check re-runs automatically. This is an automated check - no AI was used to generate this comment. |
The skills catalog index is auto-generated from SKILL.md files. Updating the github-pr-reviewer SKILL.md requires rebuilding the index or the test_index_is_up_to_date test fails. Co-authored-by: openhands <openhands@all-hands.dev>
… content The hiding step was excluding only the freshly posted result comment, so it also minimized the current cycle's acknowledgement comment and any current review object the agent posted. Now: - Track the acknowledgement comment's node_id per review cycle and exclude it alongside the result comment, so the current pair stays visible. - For PR review objects (whose node IDs aren't tracked because the agent posts them), treat the most recent automation review object as the current cycle's and skip it; only older ones are minimized. Add tests covering: previous pairs hidden, current pair kept, single review object never hidden, already-minimized skipped, non-automation comments untouched. Co-authored-by: openhands <openhands@all-hands.dev>
smolpaws
reviewed
Aug 10, 2026
smolpaws
left a comment
Contributor
There was a problem hiding this comment.
👋 I'm smolpaws — a small AI cat agent (OpenHands under the hood), reviewing on Engel's behalf since my Cloud sibling hit an error mid-run. 🐾 Second pair of eyes, not a gate.
Verdict: solid, does what was asked — one real scaling caveat. It fixes the follow-up bug correctly: the current cycle's ack comment (tracked by node_id) and result comment are excluded, and the newest automation review object is kept, so only previous pairs get minimized. minimizeComment + OUTDATED is the right API.
Main caveat — pagination cap of 100 (worth a follow-up):
_list_issue_comments, _list_pr_reviews, and the inline comments are all first: 100 with no cursor loop. The whole motivation is PRs that "accumulate a large number of comments" — exactly where >100 shows up. Issue comments are CREATED_AT ASC, so past ~100 the newest fall off the page and stop being hidden; worse, reviews(first: 100) has no ordering, so on a very active PR the current review object can fall outside the window and automation_reviews[0] picks the wrong "current" → it could hide the freshly posted review. Fine for now, but it caps the fix right at the size it targets.
Smaller notes (non-blocking):
- Review-object "current" is a heuristic, not tracked. Comments use node IDs (robust); review objects use "newest automation review by
createdAt." If one cycle posts two review objects, the older sibling of the same cycle gets hidden; if a cycle posts no review object, a previous cycle's review is wrongly kept visible. Tracking the posted review's node ID (like the ack comment) would close this. - Marker-based detection (
_is_automation_content): the code itself notes review objects "may not always carry this marker" → occasional false negatives (won't hide). Low risk, just flagging. - Tests exercise the real selection logic (
_hide_previous_automation_comments+_is_automation_content) with_minimize_comment/list calls mocked — reasonable, and good branch coverage. Missing: a >100 pagination case and a multi-review-per-cycle case.
Nice, focused change. I'd merge for the common case and file the pagination handling as a fast follow. 🐾
…o hiding scales
The hide-previous-automation step fetched issue comments and reviews with a
single first:100 call. Two problems on the comment-heavy PRs this feature
targets:
- The issue-comment query used orderBy {field: CREATED_AT} — invalid for
IssueCommentOrder (only UPDATED_AT is accepted), so GitHub returned an
error and _list_issue_comments yielded nothing, hiding no issue comments
at all. The mocked tests never exercised the live query.
- Even once valid, first:100 with no cursor caps the scan at 100 items, and
minimized comments still occupy slots (isMinimized is a field, not a
filter), so on a PR with >100 comments the recent automation pair falls
off the page and never gets hidden.
Fix: walk both connections in small pages (20). Issue comments are ordered
UPDATED_AT DESC and the scan stops on the first page with no automation
content after the automation cluster, so a normal PR costs one request and a
busy PR is walked only as far as its automation reaches. Reviews have no
orderBy argument, so they are paginated forward and sorted by the caller.
Both bounded by a max-page cap. Queries validated against the live GitHub
GraphQL API.
Adds tests for stop-early pagination, single-page, GraphQL-error, review
multi-page, and an end-to-end pass through the real _list_* helpers.
Co-authored-by: smolpaws <engel@enyst.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
GitHub struggles to render PRs that accumulate a large number of comments, and the conversation tab gets cluttered when the GitHub PR Reviewer automation runs multiple review cycles on the same PR. Each cycle posts a pair — an "OpenHands is reviewing this PR" acknowledgement comment and a review result (an issue comment and/or a PR review object) — so old cycles pile up and obscure the latest review. Hiding stale automation output keeps the conversation clean and improves the rendering experience.
Summary
outdatedreason) all content it posted in previous review cycles: earlier acknowledgement comments, earlier review-result comments, and previous PR review objects with their inline diff comments.minimizeCommentmutation with theOUTDATEDclassifier. Hidden content can still be expanded by users or unhidden by moderators.skills/github-pr-reviewer/scripts/main.py(hiding logic + ack node tracking),skills/github-pr-reviewer/SKILL.md(documented behaviour),skills/index.js(regenerated), andtests/test_github_pr_reviewer_hide.py(new tests).Issue Number
N/A
How to Test
Not directly tested end-to-end here. To validate manually:
Unit tests (
tests/test_github_pr_reviewer_hide.py) cover: previous pairs hidden, current pair kept, single review object never hidden, already-minimized skipped, and non-automation comments untouched.Video/Screenshots
N/A — this changes automation posting behaviour rather than a UI surface.
Notes
minimizeCommentworks with the sameIssues: Writepermission already required for posting comments (classic PATreposcope or fine-grained PATIssues: Read and Write).This PR was created by an AI agent (OpenHands) on behalf of a user.