Skip to content

feat: add severity floor to review prompts - #428

Merged
sv-tmueller merged 5 commits into
mainfrom
feat/407-severity-floor
Oct 1, 2026
Merged

sv-tmueller merged 5 commits into
mainfrom
feat/407-severity-floor

Conversation

@sv-tmueller

@sv-tmueller sv-tmueller commented Oct 1, 2026 •

Copy link
Copy Markdown
Owner

Closes #407

Part of batch #423

What

Adds a severity floor: five objectively checkable policy conditions make a finding must-fix, whatever the reviewer's overall read. Prompt text only, no code change.

  1. A test deleted, skipped or weakened, without the PR body saying why.
  2. --no-verify, or any other bypassed git hook.
  3. A new dependency with no justification in the PR body.
  4. A CI job with no timeout-minutes, or a workflow with no concurrency group carrying cancel-in-progress: true.
  5. A change touching the full stack, shipped without e2e.

Surfaces

  • PROMPTS in tm-review-changes.js (review, consolidate) and tm-review-codebase.js (area_review, architecture_review, consolidate), plus the matching prompts JSON, in lockstep. The Hermes renderer reads that JSON, so no separate Hermes copy exists.
  • .claude/agents/reviewer.md and .claude/adapters/prompts/reviewer.md: new ## Severity floor section.
  • Only the PROMPTS region of the workflow files is touched (the meta block belongs to Make each tm- workflow's meta a literal first statement #424).

For the human reviewer: docs/architecture/role-contracts.md

The reviewer section gains a "Severity floor" paragraph and the numbered list, after the Pass 2 paragraph. It is the host-neutral source both reviewer bindings derive from, so leaving it out would make it stale. This hunk was added beyond the issue's file list (lead decision, batch #423 Decision 2).

Decisions

  • Workers: the floor is a severity rule ("if a finding you report matches one of these"), not a detection mandate. Detection stays with the dimension briefs and the consolidate/critic stage.
  • Consolidate prompts say a floor finding cannot be downgraded, but may still be dismissed when false on the facts. The verify prompt is unchanged.
  • tm-review-codebase templates add one sentence: a whole-repo review has no diff or PR body, so apply only the conditions the current tree can show.
  • The prompt text does not cite process-core.md; condition 1 comes from the reviewer contract.

Tests

New severity-floor.test.mjs pins all five conditions in each of the 5 JS templates, the same 5 JSON keys, both reviewer.md files and the role-contracts.md reviewer section, plus the no-downgrade phrase in both consolidate templates. Red on main (17 failures), green now. npm test: 485 pass, 0 fail.

Version bumped 2.9.0 to 2.10.0 (minor). If #424 merges first, rebase and keep a version above main's.

No new dependencies.

🤖 Generated with Claude Code

sv-tmueller and others added 4 commits October 1, 2026 13:03
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Five objective policy conditions now force must-fix in the worker and consolidate templates of both review workflows. Floor sets severity, not truth: consolidate may still dismiss a floor finding that is false on the facts.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ontract

role-contracts.md is the host-neutral source both reviewer bindings derive from, so it carries the same floor to stay in sync.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Review prompts and reviewer bindings changed behavior.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@sv-tmueller sv-tmueller changed the title feat: severity floor in review prompts feat: add severity floor to review prompts Oct 1, 2026
@sv-tmueller

Copy link
Copy Markdown
Owner Author

Tester report (round 1)

VERDICT: PASS
COMMIT: 866409a
FINDINGS: none
UNTESTED CLAIMS:

  • The "may still be dismissed if false on the facts" escape in both consolidate templates (JS and JSON) is not pinned. severity-floor.test.mjs only checks the sentinel "cannot be downgraded". Scratch copy with the escape removed from codebase consolidate (JS and JSON together): npm test stays 485 pass, 0 fail.
  • The tm-review-codebase whole-repo scoping sentence in its three templates is not pinned by any test.
  • The "floor sets severity, not truth" wording in the three markdown files is not pinned. Only the five conditions are.
  • The PROMPTS.verify check in the new test only asserts condition 1 is absent. Byte-identity to main is not tested. I confirmed it by hand: verify and scout are unchanged in JS and JSON.
    Checks run:
  • npm test on the branch: 485 pass, 0 fail. node scripts/check-version-bump.mjs origin/main HEAD exits 0 (2.9.0 -> 2.10.0).
  • All 5 JS templates are changed. Each JSON key equals its JS template. Slot lists, opening 60 characters, em dashes and "fable" are unchanged or clean. adapters/prompts/reviewer.md has 0 "CLAUDE.md".
  • The diff for both workflow files touches only the PROMPTS lines (291 and 295; 267, 269 and 271).
  • Drift mutations in a scratch copy under /tmp each failed the new test and named the surface and condition. The six mutations were:
    • a reworded condition 2 in the codebase area_review JS template
    • condition 5 dropped from the changes JSON consolidate key
    • condition 3 dropped from agents/reviewer.md
    • condition 4 dropped from role-contracts.md
    • condition 1 reworded in adapters/prompts/reviewer.md
    • the changes JSON drift also tripped prompts-sync
  • Scratch merge with fix/424-literal-workflow-meta (1afc284): the only conflict is the plugin.json version line (2.10.0 vs 2.9.1). The two workflow JS files auto-merged. With 2.10.0 kept, npm test shows 495 pass, 0 fail. The merge was aborted afterwards and the worktree is clean.
    LESSONS: A sentinel-phrase assertion pins only the phrase. If a rule has a carve-out clause, pin it with its own assertion, or a consistent edit to JS and JSON can remove it unnoticed.

@sv-tmueller

Copy link
Copy Markdown
Owner Author

Reviewer report (round 1)

VERDICT: APPROVE
STAGE: quality
FINDINGS:

  1. .claude/workflows/tests/severity-floor.test.mjs:83-87, should-fix. The test pins only "cannot be downgraded". Nothing pins the dismissal escape that goes with it. The escape carries real weight in tm-review-codebase, which has no verify stage: without it, a false floor finding cannot be removed. Fix: add a second sentinel, may still dismiss it if it is false on the facts, and assert it in both consolidate templates, JS and JSON.
    Evidence: sub-plan Decision 2 ("Without the dismissal escape, a false floor finding could not be removed"). The tester's scratch copy with the escape removed still passed npm test (485 pass).
  2. .claude/workflows/tests/severity-floor.test.mjs:91-93, nit. The test is named "verify prompt is unchanged", but it only checks that condition 1 is missing from the JS verify prompt. Fix: rename it to something like "verify prompt carries no floor condition" and loop over all five conditions.
    Evidence: line 93, assert.ok(!norm(js.verify).includes(FLOOR_CONDITIONS[0])). Verify is in fact unchanged: it is context-only in the diff, in both JS and JSON.
  3. .claude/workflows/tm-review-codebase.js:267,269,271 (and the matching JSON keys), nit. No test pins the whole-repo scoping sentence. Fix (optional): assert apply only the conditions the current tree can show inside the codebase loop.
    Evidence: sub-plan Decision 3. The tester's untested claim 2.
  4. .claude/agents/reviewer.md:42-43, .claude/adapters/prompts/reviewer.md:29-30, docs/architecture/role-contracts.md:180-181, nit. "is still dismissed, with the reason" assumes the single reviewer has somewhere to record a dismissal. Its report contract has no such field (VERDICT/STAGE/FINDINGS/CHECKS/LESSONS, "nothing before or after it"), so the wording invites prose outside the contract. Fix: say the finding "is not reported", or similar, in all three files together.
    Evidence: the "Report contract" section in both reviewer.md files.
    CHECKS: n/a

@sv-tmueller
sv-tmueller marked this pull request as ready for review October 1, 2026 11:12
Resolve the plugin.json version to 2.10.0, above main's 2.9.1 from #426.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sv-tmueller
sv-tmueller merged commit 9c5eb5c into main Oct 1, 2026
2 checks passed
@sv-tmueller
sv-tmueller deleted the feat/407-severity-floor branch October 1, 2026 11:23
sv-tmueller added a commit that referenced this pull request Oct 1, 2026
Resolve the plugin.json version to 2.10.1, above main's 2.10.0 from #428.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto-must-fix severity floor in review prompts

1 participant