Verify stage in tm-review-changes, with spec-format item transforms

[sv-tmueller]

Part of batch #417

Part of #344 (package P1 of the Claude-of-Tanks adoption plan, docs/plans/344-claude-of-tanks-adoption.md on the PR #345 branch).

What to build

Nothing in tm-review-changes re-checks a finding against the tree before it reaches a human. Workers go straight to the Opus critic, which verifies, consolidates and writes the report in one pass from worker text alone. The survey's source repo refuted 10 of 24 major findings once it added an adversarial check. That is one data point, from game visuals, not code review. This package adds the check and measures our own refuted rate.

It has two halves: a spec-format extension, then a stage that uses it.

Spec format. A verify stage needs its items to be "every must-fix finding, flattened across workers, deduped, capped". items_source is a dotted path and cannot express that. Add an items_transform field naming an entry in a small, closed set of reducers that each renderer (Hermes and Codex) implements. The first entry is must_fix_deduped: flatten the worker findings, keep severity === 'must-fix', dedup on file + line + problem, then cap.

• Recommended option: a named transform. The architect confirms it or argues against it. Rejected: keeping the fan-out in JS only. The spec JSON is the portable contract (docs/architecture/adapter-interface.md), so a Hermes or Codex host would silently run a review with no verify stage. Rejected: an expression language, as over-engineering for one consumer.
• The cap follows the MAX_AREAS precedent: an args field plus items_default_cap, starting at 12.

The items_cap bug (folded in, from batch #390's follow-ups). Both renderers read args[stage.items_cap] with the literal value "args.areas", so they look up args["args.areas"] and the cap is always the default. P1's cap goes through the same line, so fix it here.

Verify stage.

• A new verify stage between review and consolidate. It runs on the worker tier, one dispatch per item, with schema { confirmed: boolean, note: string }.
• The verifier prompt states the adversarial stance outright. It starts from the position that the finding is wrong or stale, and must reproduce it against the current tree or the finding does not survive. "Cannot reproduce", "already fixed" and "the evidence does not hold" all mean confirmed: false.
• consolidate receives confirmed and refuted findings as separate inputs.
• The report schema gains a refuted array. The report carries a refuted appendix, so a human can see what the stage removed and whether it removes too much.
• Findings past the cap are reported, not dropped silently, the same way scoutDropped and ceilingReached work in tm-review-codebase.js.

These seven files must change together. Miss one and a test fails, costing a fix round.

Surface	File	Enforced by
Runtime SPEC	.claude/workflows/tm-review-changes.js	-
Portable spec	.claude/workflows/specs/tm-review-changes.spec.json	specs.test.mjs
Runtime prompts	PROMPTS const in the workflow JS	-
Portable prompts	.claude/workflows/prompts/tm-review-changes.prompts.json	prompts-sync.test.mjs
Second host	.claude/adapters/hermes-renderer.mjs (collectSlotVals, inferRole)	hermes-renderer-prompts.test.mjs
Tier pinning	stage tier field	effort-policy.test.mjs, render-path.test.mjs
Format contract	docs/architecture/adapter-interface.md "Spec format"	review

The Codex renderer needs the reducer too.

Live run. Once the tester passes, the lead (not the developer) runs the branch's tm-review-changes workflow once, on this PR's own diff, and records the results in the PR body. The owner pre-approved this run on 2026-09-28.

Acceptance criteria

• npm test is green, including specs.test.mjs, prompts-sync.test.mjs, render-path.test.mjs, effort-policy.test.mjs and hermes-renderer-prompts.test.mjs.
• A test shows must_fix_deduped filters, dedups and caps, in both the Hermes and Codex renderers.
• A test that fails on today's code shows items_cap naming args.<field> resolves to the passed value.
• A test shows a refuted finding is absent from mustFix and present in refuted.
• Findings past the cap appear in the report, not silently dropped.
• adapter-interface.md documents items_transform and the reducer set.
• One live /tm-review-changes run on this PR's diff, with confirmed count, refuted count and cost recorded in the PR body.
• The PR body records the live run's wall-clock time beside the confirmed count, refuted count, and cost (added by batch Batch: measurement-tools #417: tm-review-changes already ran past 600 s in A/B arm: live ultracode on Opus 5.5 on a replayed merged issue #405, and a verify phase makes it longer, which matters for headless judging).
• version in .claude/.claude-plugin/plugin.json is bumped.

Non-goals

• Porting the verify stage to tm-review-codebase (P2, next batch).
• An expression language in items_source.
• Codex live structured output (Codex spawn returns text).
• Deleting the dry-run stub.
• The auto-must-fix severity conditions (P3, its own issue).
• An A/B rerun with a second model.

[sv-tmueller]

Sub-plan (architect, batch #417)

SUB_PLAN (delta form). The plan holds, with these additions.

Plan source. The #406 issue body (carried from docs/plans/344-claude-of-tanks-adoption.md section 5.P1, on the PR #345 branch) plus the batch #417 P3 contract, which adds the wall-clock AC. The plan's blocker P0 has landed on main (#396, e8ec882, ctx is keyed by <stage>_result).

Size check. Fits size:M, at the top end. About 13 files change, but most edits are mirrored pairs (SPEC and spec JSON, PROMPTS and prompts JSON, the Hermes and Codex renderers). The live run is the lead's job, not part of the developer's estimate. No re-label. If it runs past one session, split it into the spec-format half (renderers, cap fix, doc) and the verify stage. Do not split it up front.

Conflicts with docs/architecture. None. adapter-interface.md is updated as part of this change. One pre-existing mismatch: the spec description names models, which the format doc forbids. Leave it alone.

Named transform: confirmed, with four refinements.

• R1. The reducer only flattens, filters and dedups. The cap stays the one generic step (items_cap / items_default_cap) and runs after the transform. That keeps one cap mechanism and computes overflow in one place.
• R2. An unknown items_transform name throws in both renderers. This is what makes option (a) better than (b): a host that lacks a reducer fails loudly instead of fanning out over unreduced items.
• R3. The verify stage descriptor is: parallelism: "dynamic-list", items_source: "review_result" (the review stage's array of worker reports), items_transform: "must_fix_deduped", items_cap: "args.<field>", items_default_cap: 12, item_label_prefix: "verify:", a VERIFY_SCHEMA, fallback: null. Add a Verify phase.
• R4. Apply the transform only after items_source resolves to an array. The existing stub-fallback path stays as it is.

Delta for the developer

• D1. A dead verifier means unverified, not refuted. parallel() null-pads, so a null verdict must never count as refuted, or a must-fix is silently dropped. Must-fix findings past the cap, or whose verifier returned nothing, go to the critic as unverified candidates and are judged as they are today. They are also listed in the report. This is my reading of "reported, not dropped": a budget cap or a dead worker must not turn the verdict into approve.
• D2. The script owns refuted, unverified and the mustFix strip. Without this, the AC "refuted finding absent from mustFix" can only be checked against model behavior.
  • Add function finalizeReport(report, refuted, unverified). It sets both arrays from the script's own data and drops from mustFix any finding whose file + line + problem key was refuted.
  • It returns a new object (same reason as the criticWithFallback comment).
  • It must not throw when the report has no mustFix array (the render-path stub returns none).
  • REPORT_SCHEMA gains refuted (FINDING fields plus note) and unverified.
• D3. Consolidate gets four inputs: confirmedFindings, refutedFindings (with the verifier's note), unverifiedFindings, and rawFindings, now limited to should-fix and nit. Those two severities are never verified, and dropping them would lose them. Overflow and dead-verifier text goes in coverageNote. The KNOWN_SLOTS entry for tm-review-changes in prompts-sync.test.mjs and the Hermes STUB_SLOTS ('[]' each) must match the final slot set exactly. Add a verify entry too (finding, diffHint).
• D4. Write mustFixDeduped(reports) and finalizeReport as function declarations, because the loadFn helper in helpers.test.mjs cannot slice arrow consts. Test them there. The reducer must skip null reports and reports with no findings array. The render-path stub has no findings today, so raw already holds undefined entries.
• D5. Add one must-fix findings entry and confirmed: true, note to fakeAgentResult in render-path.test.mjs. Otherwise verify makes zero agent() calls and its tier pin is never asserted, although the surface table says render-path enforces it. Check that the other two workflows stay green.
• D6. The items_cap fix has two parts:
  • Strip the args. prefix and coerce the value the way MAX_AREAS does (a positive integer, else items_default_cap).
  • The test that fails on main: Hermes renderWorkflow('tm-map-codebase', { areas: 1 }), using the delegate_task stub from the existing "receives the real scout areas" test (two scout areas), expects 1 area_map dispatch. Main gives 2. It needs no new export, so it fails because of the bug.
  • Codex calls execSync and cannot be stubbed. Export getDynamicListItems from both renderers and run one parametrized test over both modules. It covers filter, dedup, cap, the overflow log, an unknown name throwing, and the cap lookup.
• D7. In the renderers, overflow is a log line, like the stub-fallback log. The report-level overflow AC applies to the Claude Code runtime, the only renderer that builds consolidate inputs from runtime data (Hermes stubs those slots for every workflow). Say so in the PR body so the reviewer does not block on it.
• D8. Hermes changes:
  • collectSlotVals: a verify branch that supplies diffHint.
  • buildItemTaskPrompt: a finding slot holding the item as JSON.
  • inferRole: verify: 'fact-checker', in both renderers. It is a read-only claim audit on the worker tier; the default would be developer.
• D9. Declare verify between review and consolidate in both the SPEC and the JSON. Both renderers run stages in Object.entries order and return the last stage's result.
• D10. With zero must-fix findings, skip the verify parallel() call and log it, instead of calling it with no thunks.
• D11. Update the "Bounded by construction" comment to DIMENSIONS.length + min(must-fix count, cap) + 1. The cap is what keeps the team-guide's bounded-workflow rule true.
• D12. Fix doc drift in the same PR, or the live run's docs dimension will flag its own PR:
  • README.md:50 node label.
  • The .claude/skills/tm-review-changes/SKILL.md description.
  • Both currently say "reviewers + one critic".
  • Leave the dated docs/architecture/2026-07-05-codebase-map.md alone.
  • adapter-interface.md documents items_transform, the reducer set, and the items_cap value form args.<field>.
• D13. Do not touch criticWithFallback or safeRef. helpers.test.mjs asserts both are byte-identical across the workflow files.
• D14. Bump plugin.json 2.7.0 by a minor version. P1 and P2 also bump it; rebase on whichever merges first.

File order (write each test red first)

1. .claude/workflows/__tests__/hermes-adapter.test.mjs: the items_cap test from D6. Confirm it fails on main.
2. .claude/adapters/hermes-renderer.mjs and .claude/adapters/codex-renderer.mjs: cap fix, ITEM_TRANSFORMS, the unknown-name throw, the overflow log, the getDynamicListItems export. Put the parametrized test in a new __tests__/item-transforms.test.mjs or in the two adapter tests.
3. docs/architecture/adapter-interface.md.
4. .claude/workflows/__tests__/helpers.test.mjs: mustFixDeduped and finalizeReport tests.
5. .claude/workflows/tm-review-changes.js: SPEC, VERIFY_SCHEMA, REPORT_SCHEMA, PROMPTS (adversarial verifier text as the issue words it), the verify stage, finalizeReport, the comment.
6. .claude/workflows/specs/tm-review-changes.spec.json, .claude/workflows/prompts/tm-review-changes.prompts.json, and KNOWN_SLOTS in prompts-sync.test.mjs.
7. The rest of hermes-renderer.mjs (collectSlotVals, buildItemTaskPrompt, STUB_SLOTS, inferRole) and inferRole in codex-renderer.mjs.
8. .claude/workflows/__tests__/render-path.test.mjs: the stub from D5.
9. README.md, .claude/skills/tm-review-changes/SKILL.md, .claude/.claude-plugin/plugin.json.
10. npm test green, then the draft PR. The developer stops here.

Lead step: the live run after the tester passes

• L1. Run the branch's workflow, not the installed plugin copy (2.4.0, batch Decision 1) and not main's. When CLAUDE_PLUGIN_ROOT is set, step 2 of the /tm-review-changes skill runs ${CLAUDE_PLUGIN_ROOT}/workflows/tm-review-changes.js. So call the Workflow tool directly on the branch checkout's .claude/workflows/tm-review-changes.js.
• L2. The workers run git diff ${base}...HEAD in the session's working directory. The run needs the branch checkout as its cwd, with base: origin/main. From the main checkout it reviews the wrong diff. Confirm the output shows the Verify phase and verify:* dispatches. If there were zero must-fix findings, record that verify did not fire.
• L3. For cost, main's token-report.mjs misses workflow subagents ($4.28 reported vs $12.77 measured in A/B arm: live ultracode on Opus 5.5 on a replayed merged issue #405). Use P1's (Token report: include workflow subagents and add a per-agent wall-clock table #416) branch if it is ready, else the workflow's own subagent transcripts. Name the source for both the cost and the wall-clock figure.
• L4. Run it in the foreground. The 600 s background-task ceiling cut off a workflow in A/B arm: live ultracode on Opus 5.5 on a replayed merged issue #405, and verify adds a phase.

[sv-tmueller]

Package summary (batch #417): PR #419 ready for review

• Pipeline: architect sub-plan (delta, plan holds, top of size:M); developer (DONE_WITH_CONCERNS, deviations listed in the PR); tester PASS at 5741657 with every forwarded concern checked (mutations confirm the tests pin D1, the overflow AC and the refuted strip); lead live run; reviewer APPROVE at quality stage. No fix rounds.
• Checks at 5741657: npm test 429 pass; node scripts/check-version-bump.mjs origin/main HEAD exit 0 (2.7.0 -> 2.8.0).
• Live run (AC, lead step): confirmed 0, refuted 0, $1.76 list price, 195,901 ms measured. The verify stage did not fire, because no must-fix finding came up (D10), so the verify path is proven by tests only. The runtime refused the branch file's computed meta; the run used a hoisted-meta copy (batch Batch: measurement-tools #417 Decisions 2 and 5). Details in the PR body.
• Should-fix for the human review (recommended before merge): finalizeReport strips refuted findings from mustFix but keeps the critic's verdict, so a report can read mustFix: [] with changes-requested. The verdict still depends on the model, which D2 set out to remove. Raised by both the live run and the reviewer. Reviewer's required change: inside if (Array.isArray(report.mustFix)), after the filter, add if (out.mustFix.length === 0 && report.mustFix.length > 0) out.verdict = 'approve'; add one sentence to the function's comment; add two finalizeReport cases to helpers.test.mjs (only-refuted gives approve; refuted plus a kept finding keeps changes-requested). Exact text in the reviewer comment on PR feat: verify stage in tm-review-changes with spec-format item transforms #419.
• Nits for the human review: maxVerify missing from the invoke comment; a short line in the "Bounded by construction" comment; the raw filter throws on a null finding while mustFixDeduped skips it. Two PR-body nits were applied by the lead.
• Untested claims: a real Hermes host returning a parseable {confirmed, note} from the fact-checker role (tests stub delegate_task); Codex spawn for verify (cannot be stubbed); the verify stage live (did not fire on this diff).
• Merge note: plugin.json is also bumped by PR docs: add headless claude -p checklist to tm-ab-test #420 (Record the headless trial traps from #400, #404 and #405 in tm-ab-test #418) and PR feat: token report covers workflow subagents and adds a per-agent wall-clock table #421 (Token report: include workflow subagents and add a per-agent wall-clock table #416). Whichever merges first, the others need a one-line rebase on the version.