Token report: price Sonnet 5.5 and use the lead's real output tokens

[sv-tmueller]

Part of batch #423

What to build

The per-batch token report gets two figures wrong.

Sonnet 5.5 is unpriced. The price table has claude-sonnet-5 but not claude-sonnet-5-5, so Sonnet 5.5 usage shows up as "unpriced" and is left out of the total. PR #419 had to price it by hand (batch #417 Decision 6). Add a claude-sonnet-5-5 entry with all five rates, read from the pricing page named in the table's source on the day of the change, and update retrieved to that date.

Output tokens are estimated for everyone. The report estimates each agent's output as visible characters divided by 4. That leaves out thinking tokens and undercounts. The lead's own transcript carries the real usage.output_tokens (values in the hundreds to thousands per turn). Subagent transcripts carry only streaming placeholders (single digits to low tens), as batch #417 Decision 3 found. Use the real count for the lead's rows and keep the estimate for subagent rows. Each row, and the report notes, say which figure is measured and which is estimated.

Acceptance criteria

• The price table has a claude-sonnet-5-5 entry with all five rates, taken from the source URL, and retrieved is set to the check date.
• A fixture test shows a Sonnet 5.5 record is priced and counted in the total.
• Lead rows use the transcript's output_tokens, subagent rows keep the characters-divided-by-4 estimate, and a fixture test covers both.
• The column header and notes say which figures are measured and which are estimated. The current line that says all output is estimated is replaced.
• npm test passes, and the version in .claude/.claude-plugin/plugin.json is bumped.

Non-goals

• Re-pricing past batch reports.
• Changing how wall-clock time is worked out.
• Adding prices for models the policy doesn't pin.
• Any other way of recovering subagent thinking tokens.

[sv-tmueller]

Sub-plan (architect, batch #423)

SUB_PLAN

Plan source: none. The #425 body has scope and acceptance criteria copied from the batch #423 contract. The batch decision log has only Decision 1, which covers run mechanics and nothing about this approach. No docs/plans/ file or spec is linked. So this is the full checkpoint form.

No conflict with docs/architecture/. Those docs do not cover the token report. Batch #417 Decision 3 matches what I measured.

What I verified (read-only, counts only)

• The parser takes usage from the first line of each message id (token-report.mjs:77-101). That is safe for lead output. In two real lead transcripts, all 184 messages split over several lines repeat one output_tokens value on every line (median 529-748, max 9,541). So the first line already holds the final count.
• Subagent transcripts: median 8. About 7% of multi-line messages grow across their lines, and the tail reaches the hundreds (max 1,119). The notes should say subagent transcripts carry no reliable count. They should not say "always single digits". This is not a reason to change the subagent approach (that is a non-goal).
• Real transcripts carry both claude-sonnet-5-5 and claude-sonnet-5. Keep the old entry and add the new one.
• The chars/4 estimate is computed in two places: addToBucket (line 204) and perAgent row() (line 349). Both must read one per-record value.

Approach

1. Price table. Read https://platform.claude.com/docs/en/about-claude/pricing yourself. Add claude-sonnet-5-5 with all five rates and set retrieved to the date you read it. retrieved covers the whole table, so re-check the claude-opus-5-5 and claude-sonnet-5 rates on the same read. If any of them differ, stop and report NEEDS_CONTEXT; do not change them silently.
2. parse. parse() passes an explicit flag to parseTranscript: true for the lead, false for subagents. Do not infer it from role or agentHex. On the first line of a lead message, keep usage.output_tokens if it is a number. After the loop (at order.map, once visibleChars has built up), set:
   • outputTokens: the measured value, or else Math.round(visibleChars / 4). Rounding stays per record, so subagent sums do not change.
   • outputEstimated: true when no measured value was kept. A lead line with no output_tokens falls back to the estimate and is marked estimated, never shown as a measured 0.
   • Keep visibleChars; tests assert it. Update the comment at lines 51-55 to say the lead's output_tokens is safe to take from the first line.
3. aggregate and perAgent. emptyBucket gets outputEstimated: false. addToBucket adds record.outputTokens and sets outputEstimated ||= (the same pattern as splitMissing). perAgent row() sums r.outputTokens and sets outputEstimated: mine.some(r => r.outputEstimated). price() passes the flag through as it is, via ...bucket.
4. render.
   • In all three tables, the header Output (est.) becomes Output (est. where marked).
   • A cell whose bucket or row has outputEstimated gets (est.). This is the same convention as the Wall-clock column.
   • Mixed figures (a by-model row that covers lead and subagent calls, and the Total row) count as estimated: any estimated part marks the figure. The notes say so.
   • Replace limitations line 525 with: lead figures are measured from the lead transcript's usage.output_tokens; subagent transcripts carry no reliable count, so subagent figures are visible text plus tool-call input divided by 4, marked "(est.)"; a by-model or total figure that includes any estimate is marked too.
   • Reword line 526 so it applies to the estimate only: the estimate leaves out thinking and undercounts.
5. Doc fix in the same PR (the CLAUDE.md rule that code wins). In .claude/skills/tm-ab-test/SKILL.md:242, "the script estimates output as visible characters divided by 4" becomes stale. Change it so the script estimates subagent output that way and, before Token report: price Sonnet 5.5 and use the lead's real output tokens #425, did the same for the lead's output. Leave the A/B: native plan-and-execute vs the kickoff pipeline on real issues #400/A/B arm: live ultracode on Opus 5.5 on a replayed merged issue #405 numbers and docs/research, docs/reviews alone; past reports are a non-goal.

Files, in order (TDD: write each failing test before its code)

1. .claude/workflows/__tests__/token-report.test.mjs, new tests:
   • Sonnet 5.5: a synthetic claude-sonnet-5-5 lead line with 1M tokens in each of input, 5m write, 1h write, cache read and output. Parse, aggregate, then price against the shipped token-prices.json (not a hand-made table). Assert unpriced is empty, there is no 1h fallback note, the row cost equals the sum of the five published rates (compare with a 1e-9 tolerance, since 0.2 is a float), and pricedTotal includes it.
   • Measured vs estimated: lead-dedupe msg_AAA gives outputTokens 246 (not 738 tripled, not the 7 estimate) and is not estimated. A subagent record from agent-aaaa1111.jsonl gives the chars/4 value (not its output_tokens 50) and is estimated. In perAgent on the workflow session, the lead row is 35 and not estimated (today it shows 82), and the developer row stays 13 and is estimated.
   • Render: a lead cell has no marker, a subagent cell shows 13 (est.), a mixed by-model cell is marked, the new header is present, and the old "Output token counts are not in the transcripts" line is gone.
   • Lead line with no output_tokens: falls back to the estimate and is marked.
2. Same file, existing tests to update:
   • The hand-built record() helpers in the two aggregate describes (lines 103-115 and 147-159) have no outputTokens. With them unchanged, the test "output estimates per role add up" sums NaN, and assert.equal(NaN, NaN) passes under Object.is. That is a silent weakening. Add outputTokens and outputEstimated to the helpers.
   • Also update the header and cell regexes at 623, 624 and 635 and the limitations regex at 330. Update the assertions; do not remove them.
3. .claude/skills/tm-kickoff/token-prices.json
4. .claude/skills/tm-kickoff/token-report.mjs (parse, then aggregate/perAgent, then render)
5. Regenerate .claude/workflows/__tests__/fixtures/token-report/lead-dedupe.snapshot.md and workflow-session.snapshot.md. The second reads the real price table, so its retrieved line changes. In the diff, only Output headers and cells, the notes, retrieved and Cost cells may change. Any change to Input, Cache or Wall-clock columns is a bug. Do not edit the fixture .jsonl inputs.
6. .claude/skills/tm-ab-test/SKILL.md (the one sentence)
7. .claude/.claude-plugin/plugin.json: bump from 2.9.0; the author picks the level. The batch merge note expects a one-line version rebase.

Verification

• npm test exits 0.
• Smoke run against a real lead session, for example node .claude/skills/tm-kickoff/token-report.mjs --session <a recent orchestrai session id> --config-dir "$CLAUDE_CONFIG_DIR". Check that the lead row is unmarked and in the thousands, claude-sonnet-5-5 is no longer "(unpriced)", and subagent rows carry "(est.)". Do not paste that output into the PR.

Size check: fits size:S. About 30 changed lines in token-report.mjs, one JSON entry, roughly 40 test lines, two regenerated snapshots, one doc sentence and the version bump.

[sv-tmueller]

Package summary (batch #423): PR #427 ready for review

• Pipeline: architect sub-plan (full form, fits size:S); developer DONE, no deviations; tester PASS at 46cc9ca; reviewer APPROVE (quality stage). No fix rounds.
• Checks at 46cc9ca: npm test 478 pass, 0 fail; node scripts/check-version-bump.mjs origin/main HEAD exit 0 (2.9.0 -> 2.9.1); CI green (unit tests, version bump guard).
• Pricing: the tester re-read the pricing page on 2026-10-01. claude-sonnet-5-5 is 2 / 2.50 / 4 / 0.20 / 10 per MTok; the Opus 5.5 and Sonnet 5 rates are unchanged.
• Smoke run (developer and tester agree): the lead row shows 160,256 measured output tokens, unmarked; claude-sonnet-5-5 is priced; subagent rows carry "(est.)". The same branch script priced Make each tm- workflow's meta a literal first statement #424's live run at $1.23 (PR fix: make each workflow meta a literal first statement #426).
• Should-fix for the human review (recommended before merge): no test catches bucket.outputEstimated ||= record.outputEstimated in addToBucket turning into =. In every fixture the estimated record comes last in its bucket, so a bucket with an estimated lead turn followed by a measured one would lose its "(est.)" marker without any test failing. The shipped code is correct. Reviewer's fix: add an aggregate case where an estimated record comes before a measured record in the same bucket, and assert the bucket stays estimated. Raised by both tester and reviewer.
• Nit: .claude/skills/tm-ab-test/SKILL.md:244 is 112 characters against the file's roughly 76-character wrap; rewrap lines 242-245.
• Merge note: plugin.json is also bumped by PR fix: make each workflow meta a literal first statement #426 (2.9.1) and PR feat: add severity floor to review prompts #428 (2.10.0). Suggested order: fix: make each workflow meta a literal first statement #426, feat: add severity floor to review prompts #428, then this one, rebased above main's version.