1010# was of the wrong kind for the event, or if it was not sent, and otherwise waits a few minutes for the stub's reply
1111# naming this run attempt.
1212#
13+ # A comment command, a new pull request comment whose first line is `@cipherstash-agent review`, runs the test too,
14+ # and the verify job also checks that the comment got the review workflow's 👀 reaction. A comment always
15+ # runs the workflow file, the review workflow and the step files from the default branch, main, so this part tests
16+ # main's version, not the pull request's, and runs only once this file is on main. Comments that cannot be a comment
17+ # command start no job: other comments, comments on plain issues or closed pull requests, comments by bots or by the
18+ # review agent (the stub's own reply among them). A comment command from someone who is not a team member does start
19+ # the job, and the run ends red with `not-a-team-member`, by design: GitHub reports a team member whose organisation
20+ # membership is private as CONTRIBUTOR or NONE, so the commenter's author_association cannot tell who may ask.
21+ #
1322# Pull requests from forks (which get none of this repository's secrets) and pull requests opened by bots are not
1423# tested, and neither are drafts when they are opened or reopened: the skipped job says why. Marking a draft ready, or
1524# requesting a review from the review agent, runs the test; the review workflow takes a requested reviewer on a draft
2029# secret CLAUDE_ROUTINE_INTEGRATION_TEST_TOKEN the stub routine's bearer token
2130# variable CLAUDE_ROUTINE_INTEGRATION_TEST_TRIGGER_ID the stub routine's trigger id (`trig_…`); a variable, because a
2231# secret cannot be passed as a reusable workflow's input
23- # secrets CLAUDE_REVIEW_APP_ID, CLAUDE_REVIEW_APP_PRIVATE_KEY the membership App, used only when the pull request's
24- # author is not the review agent
32+ # secrets CLAUDE_REVIEW_APP_ID, CLAUDE_REVIEW_APP_PRIVATE_KEY the membership App, used for every comment command,
33+ # every requested-reviewer request, and a Ready request when
34+ # the pull request's author is not the review agent
2535name : Self-test
2636
2737on :
2838 pull_request :
2939 types : [opened, reopened, ready_for_review, review_requested]
40+ # Runs with this repository's secrets, from main, for any comment on an issue or pull request here. Only a team
41+ # member's comment command is acted on: the review job's condition filters out comments that cannot be one, and the
42+ # review workflow checks every rule itself, team membership included. No job checks out or runs the pull request's
43+ # code; every file comes from main. zizmor flags issue_comment because anyone who can comment can start this
44+ # workflow. Accepted for those reasons, as in examples/claude-review.yml.
45+ issue_comment : # zizmor: ignore[dangerous-triggers] only a team member's comment command is acted on
46+ types : [created]
3047
3148permissions : {}
3249
3350jobs :
3451 # Says why a pull request is not tested. Runs exactly when the review job does not, except for a review requested
35- # from anyone but the review agent or on a closed pull request, which runs neither. The review agent's login must match the review workflow's
36- # review-agent input, which this file leaves at its default; a job's condition cannot read that input. GitHub
37- # compares strings ignoring case, as the review workflow does, and a team's request has no requested_reviewer.
52+ # from anyone but the review agent or on a closed pull request, and any comment, which run neither. The review
53+ # agent's login must match the review workflow's review-agent input, which this file leaves at its default; a job's
54+ # condition cannot read that input. GitHub compares strings ignoring case, as the review workflow does, and a team's
55+ # request has no requested_reviewer.
3856 skipped :
3957 if : >-
40- (github.event.action != 'review_requested'
58+ github.event_name == 'pull_request'
59+ && (github.event.action != 'review_requested'
4160 || (github.event.requested_reviewer.login == 'cipherstash-agent' && github.event.pull_request.state == 'open'))
4261 && ((github.event.action != 'review_requested' && github.event.pull_request.draft)
4362 || github.event.pull_request.head.repo.full_name != github.repository
@@ -64,17 +83,44 @@ jobs:
6483 echo "::notice title=Self-test skipped::Not tested: $reason"
6584 echo "Self-test: skipped ($reason)" >>"$GITHUB_STEP_SUMMARY"
6685
67- # Calls this pull request's own version of the review workflow, with the stub routine's trigger and token. A draft is
68- # tested only for a requested reviewer, which the review workflow takes on drafts too.
86+ # Calls the review workflow with the stub routine's trigger and token: this pull request's own version for a
87+ # pull_request event, and main's for a comment. A draft is tested only for a requested reviewer or a comment
88+ # command, which the review workflow takes on drafts too.
89+ #
90+ # For a comment, the condition is a filter only, and lets through nothing the review workflow would skip as not a
91+ # comment command, so a red test always means something: the first line must be `@cipherstash-agent review` followed
92+ # by the end of the comment, a space, a tab or a line break (startsWith and == ignore case, as the review workflow
93+ # does). This holds for ASCII text. GitHub does not document how it compares other letters, so such a comment could
94+ # pass here and then end the run red with not-a-command. A command with other spacing, such as two spaces, is a
95+ # command to the review workflow but is not tested. fromJSON gives the tab and line breaks, which an expression's
96+ # string cannot hold. Comments by bots and by the review agent (the stub's reply among them) and comments on closed
97+ # pull requests start no job, because the review workflow skips them. There is no author_association test, as in
98+ # examples/claude-review.yml: a team member whose organisation membership is private shows as CONTRIBUTOR or NONE and
99+ # would be skipped silently. The review workflow decides, and checks team membership, so a non-member's command starts
100+ # a run that ends red with not-a-team-member, by design. A comment's payload does not say whether the pull request
101+ # comes from a fork, and the review workflow takes comment commands on forks, so they are tested: nothing from the
102+ # fork is checked out or run.
69103 review :
70104 if : >-
71- (github.event.action != 'review_requested'
105+ (github.event_name == 'pull_request'
106+ && (github.event.action != 'review_requested'
72107 || (github.event.requested_reviewer.login == 'cipherstash-agent' && github.event.pull_request.state == 'open'))
73108 && (github.event.action == 'review_requested' || github.event.pull_request.draft == false)
74109 && github.event.pull_request.head.repo.full_name == github.repository
75- && github.event.pull_request.user.type != 'Bot'
110+ && github.event.pull_request.user.type != 'Bot')
111+ || (github.event_name == 'issue_comment'
112+ && github.event.action == 'created'
113+ && github.event.issue.pull_request
114+ && github.event.issue.state == 'open'
115+ && github.event.comment.user.type != 'Bot'
116+ && github.event.comment.user.login != 'cipherstash-agent'
117+ && (github.event.comment.body == '@cipherstash-agent review'
118+ || startsWith(github.event.comment.body, '@cipherstash-agent review ')
119+ || startsWith(github.event.comment.body, format('@cipherstash-agent review{0}', fromJSON('"\t"')))
120+ || startsWith(github.event.comment.body, format('@cipherstash-agent review{0}', fromJSON('"\n"')))
121+ || startsWith(github.event.comment.body, format('@cipherstash-agent review{0}', fromJSON('"\r\n"')))))
76122 # The review workflow's jobs can hold no more than this, and GitHub refuses to start it if a job asks for more,
77- # even one that will not run. Only its react job uses it, for a comment command, which this test does not make .
123+ # even one that will not run. Only its react job uses it, to add the 👀 reaction to a comment command .
78124 permissions :
79125 issues : write
80126 uses : $/.github/workflows/claude-review.yml
@@ -85,22 +131,23 @@ jobs:
85131 CLAUDE_REVIEW_APP_ID : ${{ secrets.CLAUDE_REVIEW_APP_ID }}
86132 CLAUDE_REVIEW_APP_PRIVATE_KEY : ${{ secrets.CLAUDE_REVIEW_APP_PRIVATE_KEY }}
87133
88- # Fails unless the review workflow sent a review request and the stub replied naming this run. Runs whenever the
89- # review job ran, even if it failed, so a failure is always explained here.
134+ # Fails unless the review workflow sent a review request, the comment command (if that is what started the run) got
135+ # the 👀 reaction, and the stub replied naming this run. Runs whenever the review job ran, even if it failed, so a
136+ # failure is always explained here.
90137 verify :
91138 needs : review
92139 if : ${{ !cancelled() && needs.review.result != 'skipped' }}
93140 runs-on : ubuntu-latest
94141 timeout-minutes : 10
95- # Reads the pull request's comments. This repository is public, so the checkout below needs no contents permission,
96- # as the review workflow's own jobs show.
142+ # Reads the pull request's comments, and a comment's reactions . This repository is public, so the checkout below
143+ # needs no contents permission, as the review workflow's own jobs show.
97144 permissions :
98145 issues : read
99146 env :
100147 # Where the checkout step below puts this repository's step files, as in the review workflow's jobs.
101148 WORKFLOW_FILES : ${{ github.workspace }}/.cipherstash-workflows
102149 steps :
103- # This pull request's own steps/ directory, and nothing else. No credentials are kept.
150+ # This pull request's own steps/ directory (main's, for a comment) , and nothing else. No credentials are kept.
104151 - name : Check out the step files
105152 id : checkout
106153 uses : actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
@@ -118,7 +165,14 @@ jobs:
118165 DECISION : ${{ needs.review.outputs.decision }}
119166 SENT : ${{ needs.review.outputs.sent }}
120167 # Which event started this run, so the step can check the request was of the kind that event makes.
168+ EVENT_NAME : ${{ github.event_name }}
121169 EVENT_ACTION : ${{ github.event.action }}
170+ # The comment that started a comment command's run, whose 👀 reaction is checked; a number, or empty for a
171+ # pull_request event.
172+ COMMENT_ID : ${{ github.event.comment.id }}
173+ # The account the review workflow's react job adds the reaction as: the job's token acts as this bot. Only
174+ # its reaction counts, so a 👀 added by hand does not pass the test.
175+ REACTION_AUTHOR : github-actions[bot]
122176 # The account the stub routine posts its reply as. Only a comment by this login counts as the reply: the
123177 # job's token may not see the stub's organisation membership, so the author's standing in the repository is not used.
124178 REPLY_AUTHOR : cipherstash-agent
0 commit comments