You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 40a0989
Browse filesBrowse the repository at this point in the historyBrowse files
fix: start the self-test for a comment command whatever the commenter's association
GitHub reports a team member whose organisation membership is private as
CONTRIBUTOR or NONE, so the OWNER/MEMBER/COLLABORATOR test silently started no
run for their comment command. The review workflow's team check is the gate: a
non-member's command now starts a run that ends red with not-a-team-member, by
design. Bots, the review agent, closed pull requests and non-commands still
start no job. Tests first failed for CONTRIBUTOR and NONE commenters.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7nyyNodWTyouEMfLWr
Copy file name to clipboardExpand all lines: README.md
+4-3Lines changed: 4 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -242,7 +242,7 @@ The stub only posts a comment saying it received the request, so a test run is f
242
242
243
243
### What it does
244
244
245
-
[`.github/workflows/self-test.yml`](.github/workflows/self-test.yml) runs when a pull request in this repository is opened, reopened or marked ready for review, when someone requests a review from the review agent, `cipherstash-agent`, on a pull request here, and when a team member posts a comment command on a pull request here (see [Comment commands](#comment-commands)).
245
+
[`.github/workflows/self-test.yml`](.github/workflows/self-test.yml) runs when a pull request in this repository is opened, reopened or marked ready for review, when someone requests a review from the review agent, `cipherstash-agent`, on a pull request here, and when someone posts a comment command on a pull request here (see [Comment commands](#comment-commands)).
246
246
247
247
1. The `review` job calls `.github/workflows/claude-review.yml` by GitHub's self-repository form, `uses: $/.github/workflows/claude-review.yml`, which names this repository at the exact commit that is running. (`$/` needs github.com and runner 2.336.0 or later.) For a Ready or requested-reviewer request, that is the pull request's own version, so the version under test is the one in the pull request. For a comment command, it is `main`'s version (see [Comment commands](#comment-commands)). The job passes the stub routine's trigger id and token, never the real routine's.
248
248
2. The review workflow decides, as it would for any repository. For a review requested from `cipherstash-agent`, the person who requested it must be a member of the `@cipherstash/developers` team, as for any requested reviewer. If it decides to request a review, it sends the request to the stub. The request text includes a link to this run attempt, `<server>/<repo>/actions/runs/<id>/attempts/<n>`.
@@ -262,17 +262,18 @@ The test covers all three kinds of review request: Ready, requested-reviewer and
262
262
### Comment commands
263
263
264
264
A comment command is a new comment on a pull request whose first line is `@cipherstash-agent review` (see [Claude review](#claude-review)).
265
-
When a team member posts one on a pull request in this repository, the self-test runs as above, with these differences:
265
+
When someone posts one on a pull request in this repository, the self-test runs as above, with these differences:
266
266
267
267
-**It tests `main`, not the pull request.** GitHub always runs a comment's workflows from the default branch. So the self-test file, the review workflow it calls and the step files all come from `main`, whatever the pull request changes. To test a pull request's own version of the review workflow, use a Ready or requested-reviewer request (see [Running the test again](#running-the-test-again)). This part of the test cannot run at all until the self-test, with its `issue_comment` trigger, is merged to `main`.
268
268
- The `verify` job checks that the decision is a comment-command request for the comment that started the run. It then checks that the comment has the review workflow's 👀 reaction. The reaction must be by `github-actions[bot]` (the `REACTION_AUTHOR` setting in the workflow), the account that the review workflow's `react` job reacts as, so a 👀 added by hand does not count. This account is not yet confirmed by a live run. If the first live run fails on the reaction check, check `REACTION_AUTHOR` first. The job reads the comment's reactions every 5 seconds, for up to 60 seconds; the `react` job has finished before `verify` starts, so the reaction is normally there at once. When it finds the reaction, it shows it in a notice, `Comment command reacted to`, and in the run's summary. Last, it waits for the stub's reply for this run attempt, as for the other kinds.
269
269
- Only comments that the review workflow would take as a comment command start the test. All other comments start no self-test job at all: the workflow run shows only skipped jobs, and no `skipped` job explains it. That includes:
270
270
- ordinary comments, and comments that mention `cipherstash-agent` without being a command
271
271
- comments on plain issues, and on closed or merged pull requests
272
272
- comments by bots, and by `cipherstash-agent` itself. The stub posts its reply as `cipherstash-agent`, so its reply can never start another run.
273
-
- comments by anyone who is not an owner, member or collaborator of the repository (GitHub's `author_association`), as in [`examples/claude-review.yml`](examples/claude-review.yml). This is only a filter that saves starting a runner. The review workflow's team membership check is what decides.
274
273
- edited or deleted comments
275
274
275
+
Comments are not filtered by the commenter's standing in the repository (GitHub's `author_association`), as in [`examples/claude-review.yml`](examples/claude-review.yml), because GitHub reports a team member whose organisation membership is private as `CONTRIBUTOR` or `NONE`, and a filter on it would skip their command without any run. The review workflow's team membership check decides. So a command from someone who is not a team member does start a run, which ends red with `not-a-team-member`, by design.
276
+
276
277
The filter is a job condition, which cannot match patterns, so it takes only a first line of `@cipherstash-agent review` (in any case) followed by the end of the comment, a space, a tab or a line break. A command with other spacing, such as two spaces before `review`, is still a command to the review workflow, but it does not start the test. A unit test runs the request decision on sample comments to check that the filter never lets through a comment that the review workflow would skip as not a command.
277
278
- Comment commands are tested on drafts, because the review workflow takes them on drafts.
278
279
- Comment commands are tested on pull requests from forks too. A comment's event does not say whether the pull request comes from a fork, and the review workflow takes comment commands on forks. A comment's run has this repository's secrets even on a fork's pull request, but nothing from the fork is checked out or run: every job uses `main`'s files only, the review workflow reviews by the pull request's link, and only a team member's command is sent. So the secrets reach no code from the fork.
0 commit comments