Skip to content

Commit 38f13f4

Browse files
authored
Merge pull request #41 from cipherstash/claude/project-thread-y6piip
fix!: let the 👀 reaction through, and name the membership App by its client ID
2 parents b493c5e + 5a921c5 commit 38f13f4

8 files changed

Lines changed: 120 additions & 25 deletions

File tree

‎.github/workflows/claude-review.yml‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ on:
3232
description: The slug of the team, in the calling repository's owner organisation, whose members may request a review.
3333
type: string
3434
default: developers
35+
app-client-id:
36+
description: The client ID of the GitHub App, installed on the organisation with only "Members, read", that looks up team membership, usually `vars.CLAUDE_REVIEW_APP_CLIENT_ID`.
37+
type: string
38+
required: true
3539
review-agent:
3640
description: The login of the review agent, the account mentioned in a comment command or requested as a reviewer, whose pull requests are reviewed without a membership lookup and whose comments and reviewer requests never request a review.
3741
type: string
@@ -40,11 +44,8 @@ on:
4044
CLAUDE_ROUTINE_TOKEN:
4145
description: The code-review routine's bearer token.
4246
required: true
43-
CLAUDE_REVIEW_APP_ID:
44-
description: The id of the GitHub App, installed on the organisation with only "Members, read", that looks up team membership.
45-
required: true
4647
CLAUDE_REVIEW_APP_PRIVATE_KEY:
47-
description: That GitHub App's private key.
48+
description: The private key of the GitHub App named by `app-client-id`.
4849
required: true
4950
outputs:
5051
# For a caller that checks what happened, such as this repository's self-test (.github/workflows/self-test.yml).
@@ -142,7 +143,7 @@ jobs:
142143
continue-on-error: true
143144
uses: actions/create-github-app-token@64ab729a6294000f5aa4a4bc0b73d1f32bc45510 # v3.3.0
144145
with:
145-
app-id: ${{ secrets.CLAUDE_REVIEW_APP_ID }}
146+
client-id: ${{ inputs.app-client-id }}
146147
private-key: ${{ secrets.CLAUDE_REVIEW_APP_PRIVATE_KEY }}
147148
owner: ${{ github.repository_owner }}
148149
permission-members: read
@@ -270,9 +271,11 @@ jobs:
270271
exit 1
271272
272273
# Adds a 👀 reaction to a comment command once its request has been sent, so the requester knows it was received.
273-
# This is the only job that holds a write permission, and it sees only the decision. Reacting to a pull request's
274-
# conversation comment uses the issue-comment reactions endpoint, which GitHub documents as needing "Issues: write".
275-
# The calling job must grant `issues: write`, because a reusable workflow's jobs cannot hold more than their caller.
274+
# This is the only job that holds write permissions, and it sees only the decision. Reacting to a pull request's
275+
# conversation comment uses the issue-comment reactions endpoint. GitHub documents it as needing "Issues: write",
276+
# but on a pull request's comment it answered HTTP 403 to a token with only that, in the first live comment command
277+
# (run 44665882864), so the job also holds `pull-requests: write`. The calling job must grant both, because a
278+
# reusable workflow's jobs cannot hold more than their caller.
276279
# The checkout is given no contents permission. This relies on this repository being public, as for the decide and
277280
# fire jobs, which hold no permissions at all; it is to be confirmed by the first live run (#20).
278281
react:
@@ -282,6 +285,7 @@ jobs:
282285
timeout-minutes: 5
283286
permissions:
284287
issues: write
288+
pull-requests: write
285289
env:
286290
# Where the checkout step below puts this repository's step files.
287291
WORKFLOW_FILES: ${{ github.workspace }}/.cipherstash-workflows

‎.github/workflows/self-test.yml‎

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -29,9 +29,10 @@
2929
# secret CLAUDE_ROUTINE_INTEGRATION_TEST_TOKEN the stub routine's bearer token
3030
# variable CLAUDE_ROUTINE_INTEGRATION_TEST_TRIGGER_ID the stub routine's trigger id (`trig_…`); a variable, because a
3131
# secret cannot be passed as a reusable workflow's input
32-
# secrets CLAUDE_REVIEW_APP_ID, CLAUDE_REVIEW_APP_PRIVATE_KEY the membership App, used for every comment command,
33-
# every requested-reviewer request, and a Ready request when
34-
# the pull request's author is not the review agent
32+
# variable CLAUDE_REVIEW_APP_CLIENT_ID the membership App's client ID, and its private key, used for
33+
# secret CLAUDE_REVIEW_APP_PRIVATE_KEY every comment command, every requested-reviewer request, and
34+
# a Ready request when the pull request's author is not the
35+
# review agent
3536
name: Self-test
3637

3738
on:
@@ -123,12 +124,13 @@ jobs:
123124
# even one that will not run. Only its react job uses it, to add the 👀 reaction to a comment command.
124125
permissions:
125126
issues: write
127+
pull-requests: write
126128
uses: $/.github/workflows/claude-review.yml
127129
with:
128130
routine-trigger-id: ${{ vars.CLAUDE_ROUTINE_INTEGRATION_TEST_TRIGGER_ID }}
131+
app-client-id: ${{ vars.CLAUDE_REVIEW_APP_CLIENT_ID }}
129132
secrets:
130133
CLAUDE_ROUTINE_TOKEN: ${{ secrets.CLAUDE_ROUTINE_INTEGRATION_TEST_TOKEN }}
131-
CLAUDE_REVIEW_APP_ID: ${{ secrets.CLAUDE_REVIEW_APP_ID }}
132134
CLAUDE_REVIEW_APP_PRIVATE_KEY: ${{ secrets.CLAUDE_REVIEW_APP_PRIVATE_KEY }}
133135

134136
# Fails unless the review workflow sent a review request, the comment command (if that is what started the run) got

‎README.md‎

Lines changed: 8 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -76,7 +76,7 @@ It pins the reusable workflow to a full commit SHA, with the version in a commen
7676

7777
The example listens to `pull_request` (opened, reopened, ready for review and review requested) and `issue_comment` (created) events.
7878
A caller copied before the requested reviewer kind existed needs `review_requested` added to its `pull_request` types.
79-
Its workflow-level permissions are empty, and its calling job grants only `issues: write`, which the reaction needs: a reusable workflow's jobs can never hold more permissions than the job that calls them.
79+
Its workflow-level permissions are empty, and its calling job grants only `issues: write` and `pull-requests: write`, which the reaction needs: a reusable workflow's jobs can never hold more permissions than the job that calls them.
8080
Only the reusable workflow's `react` job uses that permission; its other jobs hold none.
8181
The calling job also has an `if:` that starts the reusable workflow only for comments on pull requests that mention `@cipherstash-agent`, and for review requests for `cipherstash-agent`, to save starting runners for every other comment and request.
8282
The reusable workflow still checks every rule itself.
@@ -96,9 +96,9 @@ The caller needs these settings, which CipherStash sets once for the whole organ
9696
| Name | Kind | What it is |
9797
| --- | --- | --- |
9898
| `CLAUDE_ROUTINE_TOKEN` | secret | The code-review routine's bearer token. |
99-
| `CLAUDE_REVIEW_APP_ID` | secret | The id of the membership GitHub App. |
10099
| `CLAUDE_REVIEW_APP_PRIVATE_KEY` | secret | The membership GitHub App's private key. |
101100
| `CLAUDE_ROUTINE_TRIGGER_ID` | variable | The routine's trigger id, which starts `trig_`. |
101+
| `CLAUDE_REVIEW_APP_CLIENT_ID` | variable | The membership GitHub App's client ID, shown on the App's settings page. It is a variable, not a secret, because it is not secret and is passed as an input. |
102102

103103
The membership GitHub App is installed on the organisation with only the "Members: read" permission.
104104
The workflow uses it to get a short-lived token and look up whether the requester (the pull request's author, the comment's author, or the person who requested the review agent as a reviewer) is in the team.
@@ -109,9 +109,10 @@ The workflow uses it to get a short-lived token and look up whether the requeste
109109
| --- | --- | --- | --- |
110110
| `routine-trigger-id` | yes | | The routine's trigger id (`trig_…`). |
111111
| `team` | no | `developers` | The slug of the team, in the calling repository's organisation, whose members may request a review. |
112+
| `app-client-id` | yes | | The membership GitHub App's client ID, usually `vars.CLAUDE_REVIEW_APP_CLIENT_ID`. |
112113
| `review-agent` | no | `cipherstash-agent` | The login of the review agent. |
113114

114-
All three secrets, `CLAUDE_ROUTINE_TOKEN`, `CLAUDE_REVIEW_APP_ID` and `CLAUDE_REVIEW_APP_PRIVATE_KEY`, are required and must be passed explicitly, as the example does.
115+
Both secrets, `CLAUDE_ROUTINE_TOKEN` and `CLAUDE_REVIEW_APP_PRIVATE_KEY`, are required and must be passed explicitly, as the example does.
115116

116117
The workflow has two outputs, for a caller that checks what happened, such as this repository's [self-test](#self-test):
117118

@@ -135,7 +136,8 @@ A run has up to three jobs:
135136
If the job could not check out its step file, no reaction is added, and the run shows a warning, but the check stays green. A failed checkout also prints its own error annotation, which is not a red check.
136137

137138
`decide` and `fire` hold no permissions.
138-
`react` holds only `issues: write`, which GitHub documents as the permission for adding a reaction to an issue or pull request comment.
139+
`react` holds only `issues: write` and `pull-requests: write`.
140+
GitHub documents `issues: write` as the permission for adding a reaction to an issue or pull request comment, but on a pull request's comment it refused a token with only that (HTTP 403), so the job also holds `pull-requests: write`.
139141
Only one `fire` runs per pull request at a time, whichever kind of review request started it: a second waits rather than cancelling the first. If a third arrives while the second is still waiting, GitHub's concurrency rules replace the second with it, and the replaced run sends nothing, and its `fire` job shows as cancelled. This is because the group uses `cancel-in-progress: false`, which keeps only one waiting run. The running one is never cancelled, and the newest request is the one that waits.
140142

141143
## The request decision
@@ -224,7 +226,7 @@ If the report's file was not checked out, the step still writes the `decision-fa
224226
If the routine call's file was not checked out, the step sends nothing and shows an error.
225227
If the reaction's file or the helper module was not checked out, or the helper module cannot be loaded, the step adds no reaction and shows a warning.
226228

227-
The `react` job holds only `issues: write`, and no `contents` permission, yet its checkout reads this repository with the job's token.
229+
The `react` job holds only `issues: write` and `pull-requests: write`, and no `contents` permission, yet its checkout reads this repository with the job's token.
228230
This relies on this repository being public, as for the `decide` and `fire` jobs, which hold no permissions at all; it is to be confirmed by the first live run (#20).
229231

230232
The routine API's address is fixed in `steps/fire.py`, so no input or event can change where the token goes.
@@ -294,7 +296,7 @@ The stub's reply is a comment on the pull request, as for the other kinds. It st
294296
| --- | --- | --- |
295297
| `CLAUDE_ROUTINE_INTEGRATION_TEST_TOKEN` | repository secret | The stub routine's bearer token. |
296298
| `CLAUDE_ROUTINE_INTEGRATION_TEST_TRIGGER_ID` | repository variable | The stub routine's trigger id, which starts `trig_`. It must be a variable, because a secret cannot be passed as a reusable workflow's input. |
297-
| `CLAUDE_REVIEW_APP_ID`, `CLAUDE_REVIEW_APP_PRIVATE_KEY` | secrets | The membership GitHub App, passed on with the same names. |
299+
| `CLAUDE_REVIEW_APP_CLIENT_ID`, `CLAUDE_REVIEW_APP_PRIVATE_KEY` | variable and secret | The membership GitHub App, passed on as the `app-client-id` input and the secret of the same name. |
298300

299301
Pull requests here are usually opened by the review agent, `cipherstash-agent`, whose Ready requests need no membership check, so the test works without the App's secrets.
300302
A pull request opened by anyone else needs the membership check.

‎examples/claude-review.yml‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,9 +12,9 @@
1212
#
1313
# Needs, as organisation or repository settings:
1414
# secret CLAUDE_ROUTINE_TOKEN the code-review routine's bearer token
15-
# secret CLAUDE_REVIEW_APP_ID the membership GitHub App's id
1615
# secret CLAUDE_REVIEW_APP_PRIVATE_KEY the membership GitHub App's private key
1716
# variable CLAUDE_ROUTINE_TRIGGER_ID the routine's trigger id (`trig_…`)
17+
# variable CLAUDE_REVIEW_APP_CLIENT_ID the membership GitHub App's client ID (`Iv…`)
1818
name: Claude review
1919

2020
on:
@@ -48,10 +48,11 @@ jobs:
4848
# command; its other jobs hold no permissions.
4949
permissions:
5050
issues: write
51+
pull-requests: write
5152
uses: cipherstash/workflows/.github/workflows/claude-review.yml@07d2f90959fcaf9d4fe64d1e9b50fe9b2a9bbac0 # feat/requested-reviewer (pre-release)
5253
with:
5354
routine-trigger-id: ${{ vars.CLAUDE_ROUTINE_TRIGGER_ID }}
55+
app-client-id: ${{ vars.CLAUDE_REVIEW_APP_CLIENT_ID }}
5456
secrets:
5557
CLAUDE_ROUTINE_TOKEN: ${{ secrets.CLAUDE_ROUTINE_TOKEN }}
56-
CLAUDE_REVIEW_APP_ID: ${{ secrets.CLAUDE_REVIEW_APP_ID }}
5758
CLAUDE_REVIEW_APP_PRIVATE_KEY: ${{ secrets.CLAUDE_REVIEW_APP_PRIVATE_KEY }}

‎steps/lookup.py‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -47,8 +47,8 @@ def api(path: str, token: str) -> tuple[int, bytes]:
4747
def look_up(token: str, org: str, team: str, requester: str) -> str:
4848
if os.environ.get("TOKEN_OUTCOME") != "success" or not token:
4949
raise LookupFailed(
50-
"Could not create the membership App's token: check the CLAUDE_REVIEW_APP_ID and "
51-
f"CLAUDE_REVIEW_APP_PRIVATE_KEY secrets, and that the App is installed on {org} with Members: read"
50+
"Could not create the membership App's token: check the CLAUDE_REVIEW_APP_CLIENT_ID variable and the "
51+
f"CLAUDE_REVIEW_APP_PRIVATE_KEY secret, and that the App is installed on {org} with Members: read"
5252
)
5353
if not (ORG_NAME.fullmatch(org) and TEAM_NAME.fullmatch(team) and LOGIN.fullmatch(requester)):
5454
raise LookupFailed(

‎steps/react.py‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@
44
which the step's continue-on-error keeps from turning the check red. A missing reaction is cosmetic: this step never
55
reports an error.
66
7-
Inputs, all environment variables: GITHUB_TOKEN (the job's token, which needs issues: write), DECISION (the request
8-
decision) and the runner's GITHUB_API_URL and GITHUB_REPOSITORY.
7+
Inputs, all environment variables: GITHUB_TOKEN (the job's token, which needs issues: write and pull-requests: write),
8+
DECISION (the request decision) and the runner's GITHUB_API_URL and GITHUB_REPOSITORY.
99
Python's standard library only (3.10 features); tests/test_react.py runs this file.
1010
"""
1111

@@ -78,7 +78,7 @@ def react(token: str, decision: str) -> None:
7878
if status not in (200, 201):
7979
raise NotReacted(
8080
f"Could not add the 👀 reaction to comment {comment} (HTTP {status}); the review was still requested. "
81-
"Check that the calling job grants issues: write"
81+
"Check that the calling job grants issues: write and pull-requests: write"
8282
)
8383
print(f"Added the 👀 reaction to comment {comment} (HTTP {status})", flush=True)
8484

‎tests/test_membership_app.py‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
"""How the review workflow names the membership GitHub App, and how its callers pass it on.
2+
3+
The App is named by its client ID, a readable organisation variable passed as an input, rather than by its app ID as a
4+
secret: actions/create-github-app-token deprecated its app-id input in favour of client-id, and an ID is not secret.
5+
Only the private key stays a secret.
6+
"""
7+
8+
import unittest
9+
10+
from harness import ROOT, SELF_TEST, _yq
11+
12+
EXAMPLE = ROOT / "examples" / "claude-review.yml"
13+
TOKEN_STEP = '.jobs.decide.steps[] | select(.id == "app-token")'
14+
15+
16+
class MembershipAppTest(unittest.TestCase):
17+
def test_the_workflow_takes_the_client_id_as_a_required_input(self) -> None:
18+
self.assertEqual(_yq('.on.workflow_call.inputs."app-client-id".type').strip(), "string")
19+
self.assertEqual(_yq('.on.workflow_call.inputs."app-client-id".required').strip(), "true")
20+
21+
def test_the_workflow_no_longer_takes_an_app_id_secret(self) -> None:
22+
self.assertEqual(
23+
sorted(_yq(".on.workflow_call.secrets | keys | .[]").split()),
24+
["CLAUDE_REVIEW_APP_PRIVATE_KEY", "CLAUDE_ROUTINE_TOKEN"],
25+
)
26+
27+
def test_the_token_step_names_the_app_by_its_client_id(self) -> None:
28+
self.assertEqual(_yq(f'{TOKEN_STEP} | .with."client-id"').strip(), "${{ inputs.app-client-id }}")
29+
self.assertEqual(_yq(f'{TOKEN_STEP} | .with | has("app-id")').strip(), "false")
30+
31+
def test_callers_pass_the_client_id_from_the_organisation_variable(self) -> None:
32+
for caller in (EXAMPLE, SELF_TEST):
33+
with self.subTest(caller=caller.name):
34+
self.assertEqual(
35+
_yq('.jobs.review.with."app-client-id"', caller).strip(), "${{ vars.CLAUDE_REVIEW_APP_CLIENT_ID }}"
36+
)
37+
self.assertEqual(_yq('.jobs.review.secrets | has("CLAUDE_REVIEW_APP_ID")', caller).strip(), "false")
38+
39+
40+
if __name__ == "__main__":
41+
unittest.main()

‎tests/test_permissions.py‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
"""Which permissions the review workflow's jobs hold, and what its callers grant them.
2+
3+
A reusable workflow's jobs can hold no more than the job that calls them, and GitHub refuses to start the workflow if
4+
any of its jobs asks for more, so the example caller and the self-test must grant exactly what the react job holds.
5+
"""
6+
7+
import pathlib
8+
import unittest
9+
10+
from harness import ROOT, SELF_TEST, WORKFLOW, _yq
11+
12+
EXAMPLE = ROOT / "examples" / "claude-review.yml"
13+
14+
# Adding a reaction to a pull request's conversation comment needs pull-requests: write. With issues: write alone,
15+
# GitHub answered HTTP 403 on the first live comment command (run 44665882864).
16+
REACT_PERMISSIONS = ["issues=write", "pull-requests=write"]
17+
18+
19+
def permissions(path: str, workflow: pathlib.Path) -> list[str]:
20+
entries = _yq(f'{path}.permissions | to_entries | map(.key + "=" + .value) | .[]', workflow).split()
21+
return sorted(entries)
22+
23+
24+
class PermissionsTest(unittest.TestCase):
25+
def test_the_react_job_can_react_to_a_pull_request_comment(self) -> None:
26+
self.assertEqual(permissions(".jobs.react", WORKFLOW), REACT_PERMISSIONS)
27+
28+
def test_the_other_jobs_hold_no_permissions(self) -> None:
29+
jobs = _yq(".jobs | keys | .[]").split()
30+
for job in jobs:
31+
if job == "react":
32+
continue
33+
with self.subTest(job=job):
34+
self.assertEqual(_yq(f".jobs.{job}.permissions | length").strip(), "0")
35+
36+
def test_the_example_caller_grants_what_the_react_job_holds(self) -> None:
37+
self.assertEqual(permissions(".jobs.review", EXAMPLE), REACT_PERMISSIONS)
38+
self.assertEqual(_yq(".permissions | length", EXAMPLE).strip(), "0")
39+
40+
def test_the_self_test_grants_what_the_react_job_holds(self) -> None:
41+
self.assertEqual(permissions(".jobs.review", SELF_TEST), REACT_PERMISSIONS)
42+
43+
44+
if __name__ == "__main__":
45+
unittest.main()

0 commit comments

Comments
 (0)