Skip to content

ci: lint workflows and test the request decision on every pull request - #10

Merged
tobyhede merged 3 commits into
mainfrom
ci/checks
Nov 18, 2026
Merged

tobyhede merged 3 commits into
mainfrom
ci/checks

Conversation

@cipherstash-agent

@cipherstash-agent cipherstash-agent commented Nov 17, 2026 •

Copy link
Copy Markdown

Summary

This adds the checks every later change to the shared review workflow depends on.
Every pull request to this repository, and every push to main, now lints the workflows and runs the request decision against saved GitHub event payloads.
The request decision is the part that reads a GitHub event and decides whether it is a review request to send on to Claude.
For now it only skips events that are not a kind of review request.

Changes

  • CI: a new Checks workflow that runs on every pull request and on every push to main. It has two jobs: actionlint, and the request decision's job, which checks the test script and then runs the saved cases. It has read-only permissions, the checkout is pinned by full SHA, and actionlint is pinned to v1.7.12 and verified by Go's module checksum database.
  • Request decision: a skeleton jq program and a shell wrapper. It returns {"decision": "skip", "reason": "not-a-review-request"} for every input.
  • Tests: request-decision/test pipes each case's input.json into the decision and compares the result with the case's expected.json. It needs only bash and jq. A case with a missing or invalid file counts as a failed case, and the run goes on to the next case. A run that finds no cases fails. There is one case: an issues event is skipped.
  • Test script check: request-decision/test-runner checks these rules for request-decision/test against a stand-in decision, so the check does not depend on what the real decision does.
  • README: explains how the checks work and how to add a test case.

The first commit on main (agent docs, glossary, licence and README) was pushed directly. The repository was empty, and a pull request needs a base branch.

Verification

  • request-decision/test: 1 case, 0 failed.
  • request-decision/test-runner: 5 checks pass. A passing case passes. No cases fails. A missing expected.json, an invalid expected.json, or a missing input.json each count as one failed case. On the test script from before the fix, 3 of these checks failed.
  • With a case expecting the wrong decision, the case fails, the difference is printed, and the run exits 1.
  • With an input that is not valid JSON, the case fails with "the decision exited with an error", and the run exits 1.
  • The workflow YAML parses. bash -n finds no syntax errors in the scripts.
  • actionlint and shellcheck were not run locally, because they are not available here. The Checks run on this pull request runs them.

Related

Review notes

One thing for #5 to settle, not this pull request: a reusable workflow runs in the calling repository, so the request decision's files are not on disk there.
#5 needs the reusable workflow to fetch them from this repository at the same commit the caller pinned, or to carry the jq program inline.

The fixes for the earlier review's suggestions (missing or invalid expected.json, no cases, the push trigger) were merged up into #11, #12 and #13 with merge commits.


Generated by Claude Code

Every later change to the review workflow changes who can cause a
review, so each rule needs a test that runs before the change merges.

The request decision reads a GitHub event and decides whether to send
it on as a review request. Its tests are saved event payloads, each
with the decision expected for it, run by a bash and jq script that
needs no network or GitHub runner, so the same tests run on a laptop
and in CI. For now the decision only skips events that are not a kind
of review request; the kinds themselves come in later tickets.

actionlint runs through `go run` at a pinned version, so Go's module
checksum database verifies what runs.

Closes #2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7ot2xooXpcuW44QqtU
…o cases

The test script stopped at the first case whose expected.json was
missing or not valid JSON, so the cases after it never ran and the
summary never printed. A run that found no cases at all passed, so a
moved or emptied cases directory would turn CI green without testing
anything.

A missing or invalid expected.json now counts as a failed case and the
run carries on. A run with no cases fails. request-decision/test-runner
checks these rules against copies of the scripts, and CI runs it.

The README and the workflow header now say the checks also run on
every push to main, and one README line holding two sentences is split.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7oR2GLajZ1RUvvkpdA
…sion

The checks of request-decision/test copied the real decision, which
reads decide.jq here but reads its program out of the reusable
workflow further up the stack. A stand-in that echoes its input keeps
the checks about the test script alone, so they run unchanged on every
branch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Jmn7oR2GLajZ1RUvvkpdA
@tobyhede
tobyhede merged commit c756fd0 into main Nov 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Set up the repository with actionlint and the request-decision test runner

3 participants