Skip to content

feat: request a Claude review from a team member's comment command - #12

Merged
tobyhede merged 40 commits into
mainfrom
feat/comment-command
Nov 19, 2026
Merged

tobyhede merged 40 commits into
mainfrom
feat/comment-command

Conversation

@cipherstash-agent

@cipherstash-agent cipherstash-agent commented Nov 18, 2026 •

Copy link
Copy Markdown

Summary

A team member can now ask for a Claude review by commenting on a pull request with a first line that starts @cipherstash-agent review.
Anything after review is passed to the review as the requester's instructions, quoted and marked as data so it cannot replace the review's own instructions.
It works on drafts, and the comment gets a 👀 reaction once the request has been sent.
Comments from people outside the developers team, from bots, and from cipherstash-agent itself are ignored, as are passing mentions, quoted commands and commands in code blocks.

Second of three stacked pull requests for #1: Ready (#11), this one, then the requested reviewer (#7).

Changes

  • Request decision: a new kind of review request, comment-command. Its rules:
    • only a newly created comment on a pull request counts;
    • the first line must start, with no leading whitespace, with the mention and review, matched without regard to case;
    • bots and the review agent are skipped before any membership lookup;
    • instructions are the rest of the comment, trimmed and capped at 2000 characters, placed in a marked block at the end of the request text.
    • A command on a closed (or merged) pull request is skipped.
    • New reasons for skipping: not-a-command, review-agent and closed.
  • Reusable workflow: a new react job adds 👀 to the comment, only after the routine request was sent and only for comment commands. Only this job holds a write permission (issues: write, the one GitHub documents for reactions on pull request comments). decide and fire keep no permissions.
  • Example caller: also listens for new comments, grants issues: write to the calling job, and, to save runner start-ups, only starts the reusable workflow for comments that mention the review agent and come from an owner, member or collaborator. That filter is for speed only: the reusable workflow's team check decides.
  • Tests: 33 new saved cases (53 in total, after dropping one that repeated another), including near-misses such as @cipherstash-agent reviewer, CRLF line endings, characters counted rather than bytes at the cap, and instructions that try to close the data block early.
  • README: the comment command, instructions, reaction, caller permission, and the default-branch limit below.

Verification

  • request-decision/test: 53 cases, 0 failed, each rule added red then green.
  • All workflow YAML parses; bash -n passes on every changed run: script.
  • Checks on the head commit: actionlint (with shellcheck) and the saved cases both pass.
  • Code review: no blocking or should-fix findings; five nits fixed (closed pull requests, the caller filter, glossary and README wording).
  • Not verified live: GitHub runs comment-triggered workflows from a repository's default branch, so the comment command can only be tried once a caller is merged into a repository's default branch. Comments from this session are written by cipherstash-agent, which is skipped by design. The react and fire jobs have no automated tests beyond YAML parsing.

Python steps (#16, #25, and the file-based layout merged up from #11)

The 👀 reaction (#16) now lives in steps/react.py and uses the shared steps/helpers.py, like the steps merged up from #11 (#25). The react job checks out this repository at the caller's pin the same way the decide and fire jobs do, and keeps only issues: write. If the checkout or a file is missing, the run shows a warning and no reaction, never a red check. Request, limits and redirect handling are unchanged.

With no inline Python left, the test that compared copied helpers is gone, and a test now fails if a Python step is ever written inline again. No Python is read out of the workflow by the tests; yq reads only the workflow's structure and its short bash wrappers. The README explains where each step lives and how to add a saved case, a unit test or a new step.

The comment command example's issue_comment trigger carries a zizmor ignore comment with its reason (#27).

Related

Review notes

  • A comment on a pull request from a fork carries no head-repository details, so it cannot be told apart from any other comment. A team member's command on such a pull request fires: the routine reviews by URL and no code from the fork runs in Actions.
  • The example caller pins the commit that added the react job (pre-release pin, redone at Release v1.0.0 and point the open rollout pull requests at it #8).

Generated by Claude Code


Generated by Claude Code

Engineers need a way to ask for a review after pushing changes, on a
draft, or with a focus, without toggling the pull request's draft state.
A new pull request comment whose first line is `@<review agent> review`
now fires a review when its author is a team member.

The first line must start with the command, so quoted replies, indented
code and fenced code blocks do not count, and near-misses such as
`reviewer` or `@cipherstash-agent-bot` do not either. Comments by bots
and by the review agent are skipped before any lookup, so a review that
mentions the agent cannot start another.

The rest of the comment is passed on as instructions, trimmed and capped
at 2,000 characters, in a block at the end of the request text that is
marked as data, so it cannot replace the routine's own instructions.

Every fire decision now says whether to react, so the reaction can be
gated on the decision rather than on the event; Ready never reacts.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
A requester needs to see that their comment command was received. The
fire job now reports whether the routine trigger accepted the request,
and a separate react job adds a 👀 reaction to the comment only then.

Only the react job holds a write permission, `issues: write`, which
GitHub documents as the permission for reactions on issue comments
(pull request conversation comments are issue comments). Ready runs
never start it, and decide and fire keep no permissions. A failed
reaction is a warning, never a red check. The comment id is read from
the decision with jq and checked to be a number before it goes into the
URL.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
The caller must listen to new issue comments for comment commands to
reach the shared workflow, and must grant `issues: write`, because a
reusable workflow's jobs can hold no more than the job that calls them.
Workflow-level permissions stay empty; only the shared workflow's react
job uses the grant.

A job-level `if:` starts the shared workflow only for comments on pull
requests that mention the review agent, which saves starting runners for
every other comment. The shared workflow still checks every rule.

The pin moves to the commit that adds the react job, since the caller
now relies on it.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Engineers and maintainers need to know what counts as a comment
command, where its instructions end up, why it works on drafts and
forks, why the caller grants `issues: write`, and that comments only
work once the caller is on the default branch. The decision's output
and the new skip reasons, `not-a-command` and `review-agent`, are
documented alongside the existing ones.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
An issue_comment payload carries no fork information, so the case differed from comment-command-member-fires only in the commenter's login and could not test fork behaviour. The README still explains that a comment command works on fork pull requests.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
A review of a closed or merged pull request is wasted work. The check runs after the command match, so ordinary comments stay not-a-command, and before the author checks, so no membership lookup is made. Covers merged pull requests, which are closed.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Adds the closed skip reason to the table, and a newer replacing request (concurrency) to the reasons a comment gets no eyes reaction.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
The glossary entry now matches the rule: the first line must start with the command.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Only owners, members and collaborators can be on the team, so other commenters need not start a runner. The comment says this is a filter, not the gate, and that the mention must match the review-agent input.

Refs #6

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Brings the Python rewrites from #11's branch up to the comment command, so
the comment command ships the tested steps rather than the shell ones they
replace:

- #14: the membership lookup step in Python, with the stub-server harness
  and its unit tests.
- #17: the decision report step in Python, so it cannot fail.
- #15: the routine trigger call in Python, and a stub server that answers
  POST requests and records their bodies.

Conflicts:

- claude-review.yml, the fire job's send step: took #11's Python step
  whole. Kept this branch's job-level `outputs: sent` and its comment (the
  Python step writes `sent=true` under the same id `send`) and the `react`
  job. The fire job's comment now says the send step's json module, not
  jq, places the decision's text in the request body.
- README.md, "What a run shows": kept #11's fuller description of `fire`
  and this branch's `react` bullet.

Both suites pass: request-decision/test (53 cases) and the unit tests (53).

Refs #14 #15 #17

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
The 👀 reaction step is about to move from shell to Python (#16). Before it
does, these tests pin down what the shell step already does, run through
the same harness as the other steps, so the switch can be shown not to
change it: 200 and 201 succeed; 403, 404, 500 and a redirect are warnings
with the status; a dropped or refused connection is a warning; a comment id
that is not all digits is a warning with no request; a large id keeps every
digit; the request is one POST of {"content":"eyes"} with the token and
GitHub's headers; and the token appears nowhere in what the step leaves.

To run it, the step gets an id (`reaction`), the harness gets warnings(),
the counterpart of errors(), and run_step sets GITHUB_REPOSITORY, which the
step builds its URL from.

Refs #16

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
The reaction was the last shell step calling an API, and the only one the
unit tests could not reach. As standard-library Python, run by the same
harness as the other steps, each answer GitHub can give is now tested.

Behaviour is unchanged: the comment id must be all digits before it goes
into the URL; one POST of {"content":"eyes"} with the job's token in a
header set in the program; 200 or 201 is success; anything else, a
timeout or a connection failure is a warning and a failed step, which
continue-on-error keeps green; no redirect is followed; the job still holds
only issues: write.

Characterising the shell step found four gaps, now closed and tested:
- curl -d sent Content-Type application/x-www-form-urlencoded; the step now
  sends application/json.
- A decision that was not JSON or not an object made jq fail the step with
  no warning; it is now the same "id is not a number" warning.
- A comment id string ending in a newline lost it to command substitution,
  so "12\n" reacted to comment 12; it is now refused.
- An empty token still sent a request; it is now a warning with none.

The timeouts are constants a test can lower, as in the send step.

Refs #16

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Brings in from #11: the membership lookup bounded in time and size
(a1a4225), the Python 3.10 unit-test job, and the send step that
counts an accepted request as sent without reading its body, with the
shared total_time_limit() and escaped() scaffolding, the
recording-redirect tests, StubServer.drip(status) and per-route
headers, and the scaffolding drift test (0309405).

Conflicts resolved:
- README.md, "Adding a unit test": kept this branch's run_step line
  (repository=, .warnings()) and #11's StubServer line (headers,
  drip(status)).
- tests/harness.py: dropped RUNNER_VARIABLES, as #11 did; nothing uses
  it. This branch's GITHUB_REPOSITORY and warnings() merged cleanly.
- The workflow merged without conflict. The lookup, final and send
  steps are byte-identical to #11's, so #11's lookup hardening holds;
  the react job is this branch's.

Semantic conflict: #11's drift test covers every step that makes a
request, which now includes react. So that the merge is green, react's
copies of the shared definitions are brought in line (TotalTimeout's
docstring, escaped(), with warning() built on it, total_time_limit()).
react's post() is unchanged here; the next commit moves it onto
total_time_limit() and hardens it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
The react step's post() armed the SIGALRM handler without restoring the
previous one, and kept the time limit running until its finally clause,
after the answer had been closed. An alarm in that window would turn a
201 into a "could not reach GitHub" warning, though the reaction was
added. post() now runs under the shared total_time_limit(), which puts
the previous handler back. It ends the limit as soon as the status is
known, on a 2xx and on an error status alike. A failure after the
status has arrived leaves that status standing.

The window is microseconds wide and the step never read the body, so
no test at the step's seam fails first. Tests for a 200 or 201 whose
body never ends, and for a 403 whose body never ends, guard the
behaviour. The redirect test now covers 301, 302, 303, 307 and 308
and points Location at a second recording server, checking that no
request and no Authorization header reaches it.

react's copy of the opener comment described the code's history; it
now matches the other steps' description of the proxy settings. The
drift test also compares error() between the steps that have it, and
the README names the copies the test checks.

Refs #16

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
cipherstash-agent pushed a commit that referenced this pull request Nov 18, 2026
Brings in from #12, by way of #11: the send step that counts an
accepted request as sent without reading its body, the shared
total_time_limit() and escaped() scaffolding and the test that keeps
every copy identical, the recording-redirect tests,
StubServer.drip(status) and per-route headers (0309405, efe1a2c), and
the react step's hardening: the previous SIGALRM handler restored, the
time limit ended as soon as the status is known, and redirect tests
over 301, 302, 303, 307 and 308 (85b0d06).

No conflicts. The workflow and README merged cleanly. The lookup,
final, send and reaction steps are byte-identical to #12's, and this
branch's requested-reviewer decision and cases are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
Brings in the request decision's move to request-decision/decision.jq, checked out by the decide job at the commit
the caller pinned (#22). The conflict was the inline REQUEST_DECISION program: this branch's version, with the comment
command, moves unchanged into decision.jq, and the workflow takes the file path from feat/ready-review-requests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the actionlint annotation change and its revert, which together change nothing, so the branches stay merged up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the decision-path test, the REQUEST_DECISION_FILE rename and the wording fixes. README conflict resolved by keeping this branch's longer scaffolding list with the new 'for now' wording.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the membership lookup and the report as step files with a shared
helper module (#23). The reaction step stays inline until #25, so the drift
test also checks its copies, including warning, against the helper module.
The harness's runner environment moved into one function, so GITHUB_REPOSITORY,
which the reaction needs, is passed through it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
cipherstash-agent pushed a commit that referenced this pull request Nov 18, 2026
… fire job

The fire job's request to the routine trigger was the last Python step written
inline, so it kept its own copies of the helpers, and the drift test had to
check them against steps/helpers.py. It now lives in steps/fire.py, which
imports the time limits, redirect blocker, annotations, output writing and
token redaction from the helper module. The routine API's address stays a
constant in the file that only the tests replace.

A job cannot see another job's files, so the fire job checks out steps/ itself,
with the same pinned actions/checkout, guards and settings as the decide job. A
test now compares the two checkouts setting by setting, so a hardening change
to one cannot miss the other. If the checkout fails, the step's own script sends
nothing, prints an error and fails, which continue-on-error keeps from turning
the check red; on #12 the react job then sees no "sent" and does nothing.

The helper's redacted() leaves text unchanged for an empty token, where the
inline str.replace would have put "[token]" between every character. Nothing
changes in practice: an empty token is refused before any request, so it never
reaches the excerpt. A test pins the helper's behaviour, and the existing
empty-token test pins that the request is never made.

Every existing routine-call test runs unchanged against the file. The drift
test now lists the inline Python steps, which on this branch is none, so a new
inline step fails until it is listed and its copies checked.

Refs #24

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in #24: the routine call runs from steps/fire.py, which the fire job
checks out the same way as the decide job. The react job is unchanged: its
reaction step stays inline until #25, so the drift test lists it as the one
inline step and still checks its copies of the shared code against
steps/helpers.py. A failed checkout in the fire job leaves "sent" unset, so the
react job does not run.

Refs #24

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the lockfile check fixes (#27).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
…react job

The reaction was the last Python step written inline, so it kept its own copies of
the helpers and needed the drift test to keep them identical to steps/helpers.py.
It now lives in steps/react.py and imports the time limits, redirect blocker and
warning from the helper module, so a hardening fix is made once.

The react job checks out steps/ itself, with the same pinned actions/checkout,
guards and settings as the decide and fire jobs, and the checkout test now covers
all three. Its permission stays issues: write only: the checkout reads this public
repository, which the decide and fire jobs already do with no permissions at all.

A missing reaction is cosmetic, so every way the file cannot run is one warning and
no reaction, never an error or a red check. The run script starts the file only
when the checkout succeeded and both react.py and helpers.py are there, like the
fire step's guard; react.py also turns a broken helper module into a warning. The
script uses exec rather than the report step's fall-through, because a reaction the
file already reported (an HTTP 403, say) would otherwise get a second, false
warning about the checkout.

Every existing reaction test runs unchanged against the file. With no inline Python
left, the drift test is gone, a test now refuses any shell: python3 {0} step, and
the harness no longer reads Python out of the workflow: yq reads only the
workflow's structure and the bash run scripts. Two tests pin that a fire job that
could not send (a failed checkout included) gives no reaction.

Refs #25

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the fire step's helpers.py guard (#24 review), the actionlint
configuration and the Ruff, mypy and Python-matrix checks (#26).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the typed step files and the Ruff and mypy fixes (#26). The drift
test that #26 tidied is kept removed, since no inline step is left. react.py
gets type hints and the same reasoned noqa comments as fire.py, and joins the
fully typed step files in pyproject.toml.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
…rmission claim

Removing `exec` from the react run script would turn every successful
reaction into the not-checked-out warning and exit 1 while all tests stayed
green. Run the real react.py through the script for 201 and 403.

That a token with only `issues: write` can check out this public repository
is not yet verified live, so the workflow comment and README now say it is
to be confirmed by the first live run (#20). README also names the step
set in test_python_steps.py and notes the checkout's own error annotation.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the mypy override test and the CI/README wording fixes.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings in the behaviour-preserving cleanups: the shared send() and
github_headers() helpers, the job-level WORKFLOW_FILES, the cached
workflow queries and the harness without its inline-step path.

Conflicts: both sides had dropped the inline-step machinery from the
harness and test_python_steps.py; the result keeps
feat/ready-review-requests's version, with this branch's react job and
exact step-file set. The react job also names its checkout as
WORKFLOW_FILES, because the harness now finds every STEP_FILE through
it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
react.post repeated the time limit, opener call, HTTPError handling and
"status already known" rule that steps/helpers.py's send() now holds,
and the GitHub API headers that github_headers() holds. It uses both,
and keeps its own message, warning and exit status unchanged: the
status still counts as soon as it arrives, and no body is read.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
test_react.py copied react.py into a temporary directory by hand and
spelled its path as ROOT/steps/react.py; it now uses step_file_copy()
and step_file_path(), and step_field() for its continue-on-error check,
as the other step tests do. test_decision_file.py's docstring now names
the react job too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
@tobyhede
tobyhede marked this pull request as ready for review November 18, 2026 23:41
@tobyhede
tobyhede added this pull request to stack #28 November 18, 2026 23:41
Keeps this branch's wording on a third request replacing the waiting one,
and adds test_react to the mypy override that now names the test files.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
…t parameter

decision.jq now builds every fire decision with one request helper, binds
command_rest once, and reads the instructions pipeline as a single
trim-and-cap step (no jq trim, so jq 1.6 still works). The saved cases are
unchanged and pass.

The test harness no longer threads a repository parameter; it always sets
GITHUB_REPOSITORY to cipherstash/example. Two react tests that repeated
other tests' coverage are merged or dropped.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
…eview

Reword the no-reaction note (a failed reaction looks the same), note that a
private organisation member may not pass the example's author_association
filter, and say that punctuation straight after review is ignored. Re-pin the
example to the latest commit that changed the decision.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr

@coderdan coderdan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

Recommendation: merge, after removing or widening the author_association filter in the example caller (finding 1), and re-pinning #13 to match.

Findings

  1. Should fix: the example's filter probably blocks every current team member. The README's own caveat says a team member with private org membership may be reported as CONTRIBUTOR or NONE, and their comment then starts no run. Today that applies to the whole team:

    • All 6 members of @cipherstash/developers have private membership. orgs/cipherstash/public_members lists none of them.
    • Fetched without authentication, their comments report CONTRIBUTOR, not MEMBER. Two examples on cipherstash/ore.rs: issue comments 6360349615 and 1253060420.
    • Event payloads are widely reported to use this same public view.

    With the example as written, a comment command from any developer would most likely start no run. The command can only be tried once a caller is on a default branch, so this would fail without any sign at rollout.

    Suggested fix: remove the author_association test. The @cipherstash-agent mention filter already skips almost every comment, and the team check is the real gate. Or add CONTRIBUTOR to the list.

    Not fully verified: I couldn't inspect a real issue_comment payload for one of these users. The first live run will confirm it either way, but the fix is cheap and removes the risk.

What I checked

  • command_rest: the mention must be at the start of the first line and is matched ignoring case. review must be followed by whitespace or the end of the line. Near-misses (reviewer, @cipherstash-agentx), quoted, indented and fenced commands, and CRLF line endings are all handled. The slice by prefix length is safe, because ascii_downcase keeps the length.
  • Instructions are trimmed and capped at 2000 characters, and sit inside a block that ends at the last >>> line, so a >>> inside the instructions cannot end the block early. Only team members can send instructions anyway.
  • Skip order: closed, bot and review-agent are all decided before any membership lookup. Edited and deleted comments, and comments on issues that are not pull requests, are ignored.
  • react: runs only when fire reports sent == 'true' and the decision has react: true. It is the only job with a write permission (issues: write), and only digits are accepted for the comment id. decide and fire stay on permissions: {}, even though the caller grants issues: write.
  • Comments on pull requests from forks: the workflow runs in the base repository and no fork code runs, so firing there is fine.
  • Saved cases: 53 of 53 pass. CI is green.

Review by Claude Code

Comment thread examples/claude-review.yml Resolved
coderdan
coderdan previously approved these changes Nov 19, 2026
…mple

GitHub reports a team member whose organisation membership is private as
CONTRIBUTOR or NONE, so the OWNER/MEMBER/COLLABORATOR test skipped their
comment commands without starting a run, and nothing told them why. A job
token on a live run also saw a non-member association for a real member. The
mention test stays as the only filter; the shared workflow's team membership
check is the gate. The README no longer documents the filter or its caveat.
The example's pin, df5be8a, is already the last commit that changed a shipped
file, so it is unchanged.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Brings the stack up to date with main.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Base automatically changed from feat/ready-review-requests to main November 19, 2026 08:11
@tobyhede
tobyhede merged commit 5fe5adf into main Nov 19, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants