Skip to content

feat(budget): per-model budget fallback chains - #1

Open
kusonooyasumi wants to merge 1 commit into
pr1-model-routingfrom
pr2-budget-fallbacks
Open

feat(budget): per-model budget fallback chains#1
kusonooyasumi wants to merge 1 commit into
pr1-model-routingfrom
pr2-budget-fallbacks

Conversation

@kusonooyasumi

Copy link
Copy Markdown
Owner

Summary

Adds per-model spend caps with fallback chains: when a model reaches its cumulative budget, agents move to a cheaper configured model instead of the whole scan stopping.

Stacked PR (fork-internal). Base is pr1-model-routing, so this diff shows only the budget-fallback changes on top of PR 1. Once PR 1 (usestrix#824) merges, this will be retargeted to usestrix/strix:main as a standalone upstream PR.

Behavior

  • No change by default — no budgets configured means no fallbacks.
  • A fallback is checked after a response returns, so a call can slightly overshoot. Strix discards that boundary response before its tool calls run, switches the agent's persisted model, and replays the session on the fallback — avoiding duplicate side effects and keeping resume on the right layer.
  • A model budget without a fallback only tracks usage; it does not stop the scan.
  • The scan-wide --max-budget-usd remains a hard stop and takes precedence.

Config

{
  "llm": {
    "model": "openai/gpt-5.4",
    "subagent_model": "z-ai/glm-4.7",
    "model_budgets_usd": { "openai/gpt-5.4": 20.0, "z-ai/glm-4.7": 10.0 },
    "model_fallbacks": {
      "openai/gpt-5.4": "z-ai/glm-4.7",
      "z-ai/glm-4.7": "deepseek/deepseek-chat"
    }
  }
}

Validated for cycles, self-references, and fallback sources that lack a budget.

Implementation notes

  • strix/core/model_routing.py (new): resolve_budget_model skips already-exhausted layers at agent create/resume; chain_uses_chat_completions_tools keeps the tool schema compatible across every layer.
  • The usage hook switches all agents sharing an exhausted model to the next layer at a response boundary.
  • Usage ledger + TUI stats attribute tokens/cost to each model actually used, including fallback models.

Test plan

  • pytest tests/test_budget_fallbacks.py — chain resolution + case-insensitive lookup, cycle/missing-budget rejection, exhausted-layer skip, cross-chain tool schema, per-model usage grouping, hydration without double-counting, and the response-boundary switch (single + multi-layer + stale in-flight).
  • Full suite green (260), ruff + mypy clean.

@kusonooyasumi
kusonooyasumi force-pushed the pr2-budget-fallbacks branch from 00fc8be to 2c24efe Compare July 21, 2026 06:39
@kusonooyasumi
kusonooyasumi marked this pull request as ready for review July 21, 2026 07:02
Add cumulative per-model spend caps that fall back to a cheaper model when a
cap is reached, instead of stopping the whole scan.

- llm.model_budgets_usd and llm.model_fallbacks (validated for cycles,
  self-references, and missing budgets) define the fallback graph.
- resolve_budget_model skips already-exhausted layers when an agent is created
  or resumed; the usage hook switches all agents sharing an exhausted model to
  the next layer at a response boundary (discarding the boundary response so
  tool calls don't double-execute).
- chain_uses_chat_completions_tools keeps the tool schema compatible across
  every layer of a fallback chain.
- Usage ledger and TUI stats attribute tokens/cost to each model actually used.
  The scan-wide --max-budget-usd remains a hard stop.
@kusonooyasumi
kusonooyasumi force-pushed the pr2-budget-fallbacks branch from 2c24efe to 7bbb669 Compare July 21, 2026 07:34
@kusonooyasumi

Copy link
Copy Markdown
Owner Author

Credential consistency for budget fallbacks

Building on the per-call credential fix in the routing PR, this PR's fallback path was updated so a fallback model re-resolves its credential rather than inheriting the exhausted model's key.

  • The shared resolve_route_api_key helper now lives in strix/core/model_routing.py (used by the child factory, the usage hook, and AgentCoordinator.transition_model).
  • When a budget fallback switches an agent's model — in both ReportUsageHooks and transition_model (which applies the switch to every matching agent) — model_settings is rebuilt with the fallback's per-call api_key. This matters for cross-provider fallbacks and same-provider-different-key setups, where the global env var can't distinguish the two.

New test test_fallback_carries_subagent_credential_per_call asserts a cross-provider fallback lands the correct key in extra_args.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant