Skip to content

fix(proxy): preserve Responses context accounting - #11

Draft
LPFchan wants to merge 1 commit into
StringKe:mainfrom
LPFchan:fix/responses-context-accounting
Draft

fix(proxy): preserve Responses context accounting#11
LPFchan wants to merge 1 commit into
StringKe:mainfrom
LPFchan:fix/responses-context-accounting

Conversation

@LPFchan

@LPFchan LPFchan commented Jul 30, 2026

Copy link
Copy Markdown

Summary / 概要

Preserve OpenAI Responses input/cache usage in the translated Anthropic stream so Claude Code can track context growth and auto-compact before the provider limit. Surface terminal context overflow as one invalid_request_error instead of silently completing the failed stream or allowing the original error to be masked by a non-streaming retry.

Related Issues / 关联 Issue

Closes #10.

Related to #5 and #6, which address the separate HTTP request-body limit encountered during large sessions.

Root Cause

The Responses translator emitted input_tokens: 0 in message_start, retained only upstream output usage, and sent only output_tokens in the final message_delta. Claude Code therefore recorded zero input usage throughout translated sessions and could not reliably reach its proactive auto-compaction threshold.

The translator also ignored response.failed, then emitted a normal end_turn. A backend context_length_exceeded event was consequently unavailable to Claude Code's reactive compaction path. In error-aware downstream variants, treating the event as a generic API failure caused Claude Code to try a non-streaming request, which ChatGPT's Codex endpoint rejects with Stream must be set to true, obscuring the overflow.

Changes

  • Propagate terminal Responses input, cache-read, cache-creation, and output usage through message_delta.usage.
  • Split cached input from the Responses total with defensive clamping, avoiding double-counting in Anthropic's additive usage model.
  • Translate context_length_exceeded to invalid_request_error; preserve unrelated failures as api_error.
  • End failed streams after the error event without fabricating block closure, usage deltas, or message_stop.
  • Add translated-SSE regression tests for ordinary usage, cache accounting, malformed fields, context overflow, and unrelated errors.

The Anthropic TypeScript SDK explicitly merges late input/cache fields from message_delta.usage, so this preserves immediate streaming and requires no buffering or token estimator.

Type of Change / 变更类型

  • Bug fix / 缺陷修复
  • New feature / 新功能
  • New provider support / 新增提供商
  • Breaking change / 破坏性变更
  • Documentation / 文档
  • Refactor / 重构

Test Plan / 测试计划

  • cargo fmt --check
  • cargo clippy --all-targets — change introduces no warnings; 12 pre-existing warnings remain on upstream main
  • cargo test responses_stream — 8 passed
  • cargo test — 346 unit tests and 1 integration test passed
  • git diff --check

Checklist / 检查清单

  • cargo fmt --check passes
  • cargo clippy — zero warnings (12 pre-existing upstream warnings remain)
  • cargo test — all tests pass
  • Documentation updated (not applicable: internal wire-compatibility fix with no new configuration or public contract)
  • No sensitive data (API keys, tokens) committed

Report terminal input/cache usage to Anthropic clients and surface Responses context overflow as an invalid request so Claude Code can compact normally.\n\nImplemented by the mutual-agreement subagent and independently reviewed and verified by the parent agent.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Responses streams report zero input usage and mask context overflow

1 participant