Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions docs-site/src/content/docs/reference/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,14 +118,18 @@ Anthropic SSE comments and `ping` events count as upstream activity, including w
thinking during context compaction. Proxy-generated keep-alives do not reset the stall deadline;
an upstream connection that stops sending activity still times out.

For Opus 5.5 through the canonical Anthropic API, ordinary routed turns default to a
128,000-token total output allowance when the caller omits a limit. This uses the model's
[documented capacity for long coding turns](https://platform.claude.com/docs/en/build-with-claude/prompt-engineering/prompting-claude-opus-5-5#calibrate-effort)
and preserves the requested reasoning effort. Adaptive thinking and answer text share the
allowance; effort does not enforce a separate thinking budget. Explicit limits remain
authoritative, and other models and custom Anthropic-compatible endpoints retain their defaults.

Routed context compaction for the same model and endpoint uses at most
Ordinary routed turns use known provider/model output allowances by default, including
requests with smaller caller-supplied caps. Set the global `outputTokenPolicy` to `request`
to honor explicit caps and configured fallback budgets. Output metadata is matched to the
provider destination before selecting an allowance.

Native Claude Fable 5.2 and Opus 5.5 have 128,000-token output allowances. Adaptive thinking
and answer text share the allowance, while effort controls reasoning depth. Protocols
that expose no output-budget field leave generation limits to the upstream service.
See [output token allowances](/reference/configuration/providers/#output-token-allowances)
for configuration and coverage.

Routed context compaction for Opus 5.5 through the canonical Anthropic API uses at most
`medium` reasoning effort and defaults to a 64,000-token total output limit when the caller
omits one. Adaptive thinking and summary text share that limit; effort does not reserve a
fixed number of tokens for the summary. Explicit output limits and lower reasoning efforts
Expand Down
30 changes: 28 additions & 2 deletions docs-site/src/content/docs/reference/configuration/providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ authenticated.
| `disabledModels?` | `string[]` | — | Models hidden from Codex's catalog and `/v1/models`, but not blocked from direct proxy calls. A routed id is removed from listings. An account-qualified native id hides only that selector row; a bare native GPT id hides the bare row and every account-selector row for that model. The dashboard Models page exposes only routed and bare native rows; use this configuration field directly to hide one selector-qualified row. |
| `providerContextCaps?` | `Record<string, number>` | `{}` | Per-provider Codex-visible context caps. A cap only lowers a known context window. |
| `contextCapValue?` | `number` | `350000` | Value used by the dashboard context-cap controls; changing it updates every enabled `providerContextCaps` entry. |
| `outputTokenPolicy?` | `"request" \| "model-max"` | `"model-max"` | Uses the known output allowance for ordinary requests, including requests with a smaller caller-supplied cap. Select `request` to honor caller caps and configured fallback budgets. |
| `codexAccounts?` | `CodexAccount[]` | `[]` | ChatGPT/Codex pool account metadata managed by Codex Auth. Secrets live separately in `codex-accounts.json`. |
| `pausedCodexAccountIds?` | `string[]` | `[]` | Accounts excluded from Pool selection until resumed, including the main `__main__` account when paused. |
| `codexAccountNamespaces?` | `Record<string, string>` | — | Optional map from an arbitrary public model selector to a stored Codex account target. Each selector whose target is present adds separate `<selector>/<native-openai-model>` rows to the Codex picker; each row uses only that account. With any selector active, bare native rows are hidden in the picker, but their ids remain routable and listed by raw `/v1/models` unless explicitly disabled. |
Expand All @@ -38,6 +39,31 @@ configured pool ids or selector targets also cannot reuse a selector. Keep raw a
emails private; the selector is the public name. See [Routing Configuration](/reference/configuration/routing/)
for exact-selection behavior and precedence.

## Output token allowances

Ordinary requests use the known output allowance for their selected provider and model by
default. This also applies when an existing configuration omits `outputTokenPolicy`.

```json
{
"outputTokenPolicy": "request"
}
```

With the default `model-max` policy, a smaller caller-supplied output cap is raised when CodexCommander has
matching output-limit metadata for the route. For example, a 32,768-token request for native
Claude Fable 5.2 can use its 128,000-token allowance. Thinking and visible output share that
allowance.

The `request` setting shown above honors explicit caller caps. Requests that omit a cap can
use known model allowances after configured fallback budgets. Unknown capacities use the
request or adapter fallback. Adapters whose upstream protocol has no output-limit field use
the upstream's allowance.

Context compaction keeps its maintenance budgets. Provider output ceilings and context
windows still apply, and reaching a ceiling produces an incomplete response. A larger
allowance can permit longer generation and higher usage.

## Reserved OpenAI providers

`openai` and `openai-apikey` are fixed reserved ids. `openai.codexAccountMode` is `"pool"` by default
Expand Down Expand Up @@ -67,8 +93,8 @@ metadata, and Pro virtual ids rewrite to the base wire model with `reasoning.mod
| `modelContextWindows?` | `Record<string, number>` | Per-model context caps. These override `contextWindow` and never raise smaller live metadata. |
| `modelInputModalities?` | `Record<string, string[]>` | Per-model input hints such as `["text"]` or `["text", "image"]`. |
| `modelMaxInputTokens?` | `Record<string, number>` | Positive per-model max input limits used for catalog auto-compaction hints. |
| `defaultMaxOutputTokens?` | `number` | Provider-wide `openai-chat` fallback when the client omits `max_output_tokens`. |
| `modelMaxOutputTokens?` | `Record<string, number>` | Positive per-model `openai-chat` fallback budgets; exact/pattern matches beat the provider default. |
| `defaultMaxOutputTokens?` | `number` | Provider-wide fallback for translated Anthropic, Chat Completions, Google, and Command Code requests when the client omits an output cap. |
| `modelMaxOutputTokens?` | `Record<string, number>` | Positive per-model fallback budgets for translated requests. Exact/pattern matches beat the provider default. The `model-max` policy uses known model allowances ahead of these budgets. |
| `chatCompletionTokenField?` | `"max_tokens" \| "max_completion_tokens"` | Chat Completions output-budget field. Defaults to `max_tokens`; canonical Kimi Coding Plan presets use `max_completion_tokens`. |
| `headers?` | `Record<string, string>` | Extra upstream headers. Authorization, cookies, API-key headers, embedded newlines, and invalid names are rejected. |
| `openRouterRouting?` | `OpenRouterProviderRouting` | Default OpenRouter `order`, `only`, and `allowFallbacks` preferences; valid only for canonical OpenRouter with `openai-chat`. |
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "codexcommander",
"version": "0.1.26",
"version": "0.1.27",
"private": true,
"description": "CodexCommander — universal provider proxy for OpenAI Codex & Claude Code — use any LLM with Codex CLI/App/SDK and Claude Code",
"type": "module",
Expand Down
26 changes: 14 additions & 12 deletions src/adapters/anthropic.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import type { IncomingMeta, ProviderAdapter } from "./base";
import { debugDroppedFrame } from "../lib/debug";
import { isCanonicalAnthropicEndpoint } from "../lib/anthropic-endpoint";
import { knownModelOutputTokenLimit, resolveOutputTokenBudget } from "../output-budget";
import type {
AdapterEvent,
CodexCommanderAssistantMessage,
Expand Down Expand Up @@ -39,7 +40,7 @@ function toAnthropicContentPart(p: CodexCommanderContentPart): unknown {

/** Default `max_tokens` when Codex omits `max_output_tokens`. */
const DEFAULT_MAX_TOKENS = 8192;
/** Safe ceiling for `max_tokens` (thinking + visible output) across current Claude 4.x models. */
/** Compatibility fallback when a legacy thinking model has no verified output limit. */
const REASONING_MAX_TOKENS_CEILING = 32_000;
/** Ordinary adaptive-thinking sizing: heuristic max effort allowance (32k) + 8192.
* Adaptive effort is soft guidance, so this total does not reserve a hard visible-output portion. */
Expand Down Expand Up @@ -857,11 +858,8 @@ export function createAnthropicAdapter(provider: CodexCommanderProviderConfig, c
enforceAnthropicImageLimits(messages);
const tools = toolsToAnthropicFormat(parsed, toolNames);

// Adaptive effort has no fixed thinking reservation: the legacy effort heuristic
// can exhaust its shared total before an ordinary Opus 5.5 turn finishes. Use the
// documented long-turn allowance only for this known model and native endpoint;
// compatible gateways may have lower limits. Compaction keeps its smaller summary
// allowance and effort cap, while an explicit caller total always wins.
// Native Opus maintenance keeps its smaller summary allowance and effort cap.
// Ordinary turns resolve their total independently from reasoning effort.
const version = claudeFamilyVersion(parsed.modelId);
const nativeOpus55 = version?.family === "opus" && version.major === 5 && version.minor === 5
&& isCanonicalAnthropicEndpoint(provider.baseUrl);
Expand All @@ -871,8 +869,8 @@ export function createAnthropicAdapter(provider: CodexCommanderProviderConfig, c
&& requestedEffort !== "none" && requestedEffort !== "minimal" && requestedEffort !== "low"
? "medium"
: requestedEffort;
const maxOutputTokens = parsed.options.maxOutputTokens
?? (nativeOpus55 ? (opusCompaction ? OPUS_5_5_COMPACTION_MAX_TOKENS : OPUS_5_5_MAX_TOKENS) : undefined);
const maxOutputTokens = resolveOutputTokenBudget(provider, parsed,
nativeOpus55 ? (opusCompaction ? OPUS_5_5_COMPACTION_MAX_TOKENS : OPUS_5_5_MAX_TOKENS) : undefined);
// Do not mutate parsed.options: requested-effort diagnostics describe the client.
// Anthropic has no reasoningLog field for output_config.effort in the shared contract.

Expand Down Expand Up @@ -927,12 +925,16 @@ export function createAnthropicAdapter(provider: CodexCommanderProviderConfig, c
// visible output) and budget_tokens >= 1024. Codex sends the SAME value for both, which
// 400s ("max_tokens must be greater than thinking.budget_tokens"). Size them so max_tokens
// always exceeds the budget within a model-safe ceiling, reserving room for visible output.
const maxOut = maxOutputTokens ?? DEFAULT_MAX_TOKENS;
const wantBudget = reasoningBudget(reasoningEffort);
const maxTokens = Math.min(REASONING_MAX_TOKENS_CEILING, Math.max(maxOut, wantBudget + OUTPUT_HEADROOM));
const budget = Math.max(MIN_THINKING_BUDGET, Math.min(wantBudget, maxTokens - OUTPUT_FLOOR));
const knownLimit = knownModelOutputTokenLimit(provider, parsed.modelId);
const maxTokens = knownLimit !== undefined && maxOutputTokens !== undefined
? maxOutputTokens
: Math.min(REASONING_MAX_TOKENS_CEILING, Math.max(maxOutputTokens ?? DEFAULT_MAX_TOKENS, wantBudget + OUTPUT_HEADROOM));
body.max_tokens = maxTokens;
body.thinking = { type: "enabled", budget_tokens: budget };
if (maxTokens > MIN_THINKING_BUDGET) {
const budget = Math.max(MIN_THINKING_BUDGET, Math.min(wantBudget, maxTokens - OUTPUT_FLOOR));
body.thinking = { type: "enabled", budget_tokens: budget };
}
}
// Extended thinking disallows temperature != 1 and top_p — drop both or the API 400s.
delete body.temperature;
Expand Down
3 changes: 2 additions & 1 deletion src/adapters/command-code.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { dispatchHttpFetch, cleanupResponseDispatch, closeAdapterObservation, observeAdapterEvent, observeDispatch, responseDispatch } from "../usage/dispatch-http";
import { resolveOutputTokenBudget } from "../output-budget";
import { randomUUID } from "node:crypto";
import { execFile as execFileCallback } from "node:child_process";
import { promisify } from "node:util";
Expand Down Expand Up @@ -356,7 +357,7 @@ export function createCommandCodeAdapter(provider: CodexCommanderProviderConfig)
messages: wireMessages(parsed.context.messages),
tools: wireTools(tools),
system,
max_tokens: parsed.options.maxOutputTokens ?? provider.defaultMaxOutputTokens ?? 64_000,
max_tokens: resolveOutputTokenBudget(provider, parsed, provider.defaultMaxOutputTokens ?? 64_000),
// The proprietary /alpha/generate endpoint is NDJSON-stream-only; the proxy converts
// the buffered/streamed events to the client's requested shape (parsed.stream).
stream: true,
Expand Down
4 changes: 3 additions & 1 deletion src/adapters/google.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { AdapterFetchContext, AdapterRequest, ProviderAdapter } from "./base";
import { resolveOutputTokenBudget } from "../output-budget";
import { debugDroppedFrame } from "../lib/debug";
import { createHash } from "node:crypto";
import { createImageBudget, materializeInlineImage, MAX_ENCODED_BYTES_PER_IMAGE, artifactHttpUrl } from "../images/artifacts";
Expand Down Expand Up @@ -335,7 +336,8 @@ export function createGoogleAdapter(provider: CodexCommanderProviderConfig): Pro
if (toolConfig) body.toolConfig = toolConfig;

const generationConfig: Record<string, unknown> = {};
if (parsed.options.maxOutputTokens) generationConfig.maxOutputTokens = parsed.options.maxOutputTokens;
const maxOutputTokens = resolveOutputTokenBudget(provider, parsed);
if (maxOutputTokens !== undefined) generationConfig.maxOutputTokens = maxOutputTokens;
if (parsed.options.temperature !== undefined) generationConfig.temperature = parsed.options.temperature;
if (parsed.options.topP !== undefined) generationConfig.topP = parsed.options.topP;
if (parsed.options.stopSequences) generationConfig.stopSequences = parsed.options.stopSequences;
Expand Down
6 changes: 4 additions & 2 deletions src/adapters/openai-chat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import type { AdapterRequest, ProviderAdapter } from "./base";
import type { AdapterEvent, CodexCommanderAssistantMessage, CodexCommanderContentPart, CodexCommanderMessage, CodexCommanderParsedRequest, CodexCommanderProviderConfig, CodexCommanderTextContent, CodexCommanderThinkingContent, CodexCommanderToolCall, CodexCommanderUsage } from "../types";
import { isAllowedToolChoice, modelInList, namespacedToolName, resolveToolChoiceWireName, toolAllowedByChoice } from "../types";
import { mapReasoningEffort, modelRecordValue } from "../reasoning-effort";
import { resolveOutputTokenBudget } from "../output-budget";
import { debugProviderDiagnostic } from "../lib/debug";
import { isDebugEnabled } from "../lib/debug-settings";
import { isCyberPolicyCode } from "../lib/errors";
Expand Down Expand Up @@ -881,7 +882,8 @@ export function createOpenAIChatAdapter(provider: CodexCommanderProviderConfig):
stream: parsed.stream,
};
if (modelInList(provider.reasoningSplitModels, parsed.modelId)) body.reasoning_split = true;
const maxTokens = resolveMaxTokens(provider, parsed);
const requestedMaxTokens = resolveMaxTokens(provider, parsed);
const maxTokens = resolveOutputTokenBudget(provider, parsed, requestedMaxTokens);
const openRouterRouting = resolveOpenRouterRouting(provider, parsed.modelId);
if (openRouterRouting) body.provider = openRouterProviderPayload(openRouterRouting);
if (tools) body.tools = tools;
Expand Down Expand Up @@ -921,7 +923,7 @@ export function createOpenAIChatAdapter(provider: CodexCommanderProviderConfig):
wireValue: reasoningEffort,
};
} else if (modelInList(provider.thinkingBudgetModels, parsed.modelId)) {
const budget = thinkingBudgetForEffort(parsed, reasoningEffort, maxTokens);
const budget = thinkingBudgetForEffort(parsed, reasoningEffort, requestedMaxTokens);
if (budget !== undefined) {
body.thinking_budget = budget;
reasoningLog = {
Expand Down
6 changes: 6 additions & 0 deletions src/adapters/openai-responses.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { isNativeResponsesProvider, NATIVE_RESPONSES_HEADERS } from "../responses/native-policy";
import { knownModelOutputTokenLimit, resolveOutputTokenBudget } from "../output-budget";
import { createHash } from "node:crypto";
import type { AdapterRequest, IncomingMeta, ProviderAdapter } from "./base";
import { namespacedToolName, type AdapterEvent, type CodexCommanderParsedRequest, type CodexCommanderProviderConfig, type CodexCommanderUsage } from "../types";
Expand Down Expand Up @@ -1235,6 +1236,11 @@ export function createResponsesPassthroughAdapter(provider: CodexCommanderProvid
);
outBody = normalizeImageGenClientTools(outBody);
}
if (!forward && provider._outputTokenPolicy === "model-max"
&& knownModelOutputTokenLimit(provider, parsed.modelId) !== undefined
&& parsed._compactionRequest !== true && isPlainObject(outBody)) {
outBody = { ...outBody, max_output_tokens: resolveOutputTokenBudget(provider, parsed) };
}
if (!native && (forward || parsed._previousResponseInputExpanded === true)) {
outBody = repairOversizedReplayCallIds(outBody);
}
Expand Down
2 changes: 2 additions & 0 deletions src/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1107,6 +1107,7 @@ const configSchema = z.object({
.optional(),
providers: z.record(z.string(), providerConfigSchema),
defaultProvider: z.string().min(1),
outputTokenPolicy: z.enum(["request", "model-max"]).optional(),
claudeCode: claudeCodeSchema.optional(),
clientIntegrations: clientIntegrationsSchema.optional(),
subagentModels: subagentRosterSchema.optional(),
Expand Down Expand Up @@ -2524,6 +2525,7 @@ export function getDefaultConfig(): CodexCommanderConfig {
},
},
defaultProvider: "openai",
outputTokenPolicy: "model-max",
subagentModels: DEFAULT_SUBAGENT_MODELS.map(model => ({ model })),
multiAgentGuidanceEnabled: true,
websockets: false,
Expand Down
Loading
Loading