fix(cost): accumulate cost on resume and use final result-event cost - #573
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #541
Problem
Two independent bugs both caused a task's recorded cost (and token usage) to be undercounted, which is what the reporter saw: after clicking Resume, the new run's cost showed on the run but "not updated elsewhere" — the analytics totals (
/api/analytics/costs, which sumstasks.cost_usd) lost spend.Bug 1 — resume/restart overwrote cost instead of accumulating
On a resumed or force-restarted run,
task-workerstreams the agent's logs into a freshallLogsbuffer, soadapter.parseResult(...)returns only that invocation'stotal_cost_usd/ tokens — aclaude --resume(or a brand-new session on the existing PR branch) is a separate process with no knowledge of what the prior run spent. The worker then did a plaindb.update(tasks).set({ costUsd: String(result.costUsd) }), replacing the original run's cost with just the resumed invocation's spend. Since analytics sumstasks.cost_usd, every resume silently dropped the earlier spend.Fix: accumulate
prior + currenton continuation runs. Because each relaunch is a distinct process reporting only its own cost,prior + currentis always the true total and never double-counts. A genuine first run (no continuation signal) still writes its value directly, preserving replace-from-scratch semantics.Continuation is detected from the job data:
resumeSessionId(/resume,--resume),restartFromBranch(/force-restartand reconciler auto-resume on the existing PR), orresumePrompt(present on every relaunch path — including message-resume, where the stored session id may be absent).inputTokens/outputTokenshad the same overwrite bug and are accumulated too (integers, exact).Cost is stored as a string (to avoid float drift), so the addition goes through a new decimal-safe helper
addCostStrings(a, b)(packages/shared/src/utils/cost.ts) that scales both operands to integers at the finer precision, adds, and re-emits a plain decimal literal (no scientific notation) soCAST(cost_usd AS NUMERIC)still parses it.Bug 2 — cost parser took the first
total_cost_usdClaudeCodeAdapter.parseResultusedlogs.match(/"total_cost_usd":\s*([\d.]+)/)— the first match. In--input-format stream-jsonmode, a multi-turn run (mid-task user messages) emits aresultevent per turn, and each result'stotal_cost_usdis the cumulative cost for the process so far. Taking the first match froze the cost at turn one and dropped all later spend.Fix: take the cost from the last
resultevent (captured in the existing event loop, which already tracks "last result event wins"), falling back to the last global regex match only when a result line isn't valid JSON. Token parsing already sums per-message assistantusage, which is correct, so it was left as-is.Note on the reporter's "continually update" expectation
Live, mid-run cost updates are not feasible for Claude Code: cost only arrives in the terminal
resultevent, never incrementally during a turn. This PR therefore fixes the correctness of the final recorded cost (and its accumulation across resumes), not live streaming of a running cost. A running task will still only show cost once its current turn's result event lands.Resume vs restart handling (accounting semantics)
Both resume and restart launch a fresh Claude process that reports only its own spend, so accumulating
prior + currentis the accurate total in both cases and cannot double-count. Only a genuine first run (none ofresumeSessionId/restartFromBranch/resumePrompt) overwrites. OpenTelemetry cost/token metrics continue to record the per-run delta (correct for additive counters).Tests
packages/agent-adapters/src/claude-code.test.ts: multi-turn run assertsparseResultreturns the last result-event cost, not the first; plus a non-JSON fallback case asserting the last regex match wins.packages/shared/src/utils/cost.test.ts: unit tests foraddCostStrings(accumulation, null/empty prior, decimal-safety0.1 + 0.2 -> 0.3, mixed precision, no scientific notation, repeated resumes) andaddTokenCounts.apps/api/e2e/repo-task.e2e.test.ts: end-to-end resume through the real pipeline + fake runtime — first run records0.05, resume adds0.03, task total becomes0.08(tokens100->200/25->50).Verification
cd apps/api && npx tsc --noEmit— cleanpnpm turbo typecheck— 12/12cd packages/agent-adapters && npx vitest run— 241 passedcd packages/shared && npx vitest run— 452 passedcd apps/api && npx vitest run— 2167 passedpnpm --filter @optio/api test:e2e repo-task— 5 passed (run twice, no flake)pnpm format:check— clean