Skip to content

feat(proxy): route mode for weighted multi-provider routing - #522

Open
AtaraxiaEpiphany wants to merge 16 commits into
SaladDay:mainfrom
AtaraxiaEpiphany:feat/route-mode
Open

feat(proxy): route mode for weighted multi-provider routing#522
AtaraxiaEpiphany wants to merge 16 commits into
SaladDay:mainfrom
AtaraxiaEpiphany:feat/route-mode

Conversation

@AtaraxiaEpiphany

@AtaraxiaEpiphany AtaraxiaEpiphany commented Oct 17, 2026

Copy link
Copy Markdown

Summary

Adds route mode: proxy-level weighted multi-provider routing, mutually exclusive with automatic failover. While route mode is active for an app, incoming proxy requests are distributed across a configured group of member providers by smooth weighted round-robin, with per-member model mapping and health-check integration.

What it does

  • Storage: SQLite-backed route group (one per app): ordered members, each with a weight (1–1000), enabled flag, and exact request-model → actual-model mapping map. Corrupt or structurally invalid stored config fails closed to disabled.
  • Selection: smooth weighted round-robin — deterministic 3:1 weights yield exactly 3:1 distribution in any window, no clumping. Dangling or disabled members are dropped at selection time.
  • Per-member model mapping: the request model is rewritten to the member's mapped model before forwarding; unmapped models pass through verbatim. Applied per member, per request.
  • Health checks: members participate in the existing circuit-breaker/health machinery; unhealthy members are skipped, all-open fails closed with a clear error.
  • Exclusivity guards: route mode and automatic failover refuse to be enabled while the other is active; managed-session disable is refused while route mode is on. Unsupported apps (anything without proxy takeover: claude/codex/gemini only) are rejected with a clear error.
  • TUI: new Route Mode screen and interactive editor (add/remove/reorder members, weights, per-member mappings, enable/disable), contextual ? help, request-model picker, targeted refresh.
  • Lifecycle: enable requires an active daemon-managed proxy session; disabling preserves member config and leaves proxy takeover untouched.

Notes for reviewers

  • History is linear: rebased onto current main (no merge commits).
  • Route mode engage path reuses the existing failover ordered-candidate replay, so in-request failure behavior (skip to next member) is unchanged machinery.
  • Unsupported apps are rejected at the service layer (enable_route_mode_for_app / disable_route_mode_for_app), not just hidden in the TUI.

Test evidence

  • cargo fmt --check clean, cargo clippy 0 errors/warnings on the branch.
  • Full test suite: 5236 passed; 11 failures are pre-existing on upstream main (verified by control run on a clean main worktree: same 11 fail there — sync/prompt/usage/managed-session probe tests, unrelated to this branch).
  • New integration target src-tauri/tests/proxy_route_mode.rs (~985 lines): enable guards, exclusivity, weighted 3:1 distribution over 40 requests, per-member mapping rewrite, failover-between-members applying each member's own mappings, startup recovery of flag + takeover intent, codex mapping path.
  • End-to-end wire verification (outside the test suite): built binary in a sandboxed config dir, proxy in route mode with members weighted 3:1, traffic captured via mitmproxy reverse listeners — 20/20 requests succeeded, upstream distribution exactly 15/5 (3:1), member A's mapped model and member-specific API-key rewrite observed on the wire, path and headers passed through verbatim.
  • Environment note for anyone running the suite in a container: tests must run under umask 022; the default container umask 002 makes temp config dirs group-writable and trips the app's managed-config-directory permission check (17 spurious lib-test failures).

🤖 Generated with Claude Code

Hannibal and others added 16 commits October 17, 2026 10:46
Hannibal @claude
feat(db): route-mode group storage
7e33d0a 
Store per-app route-mode groups (weighted provider pools) as JSON
documents in the existing settings KV table under route_mode_{app};
this fork's schema stays synced with upstream, so no tables or columns
are added and SCHEMA_VERSION is unchanged.

- RouteModeGroup/RouteModeMember with camelCase serde and defaults
- get_route_mode_group fails closed to a disabled default on missing,
  corrupt, or structurally invalid data
- save_route_mode_group validates structure and provider existence
  under one connection lock
- route_mode_enabled_for_app / set_route_mode_enabled flag helpers
- unit tests: roundtrip, flag-flip isolation, every validation
  rejection, corrupt-JSON fail-closed

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(proxy): smooth weighted provider selection
26a92a9 
Route mode branch in ProviderRouter::select_providers, consulted
before the failover queue: while the app's route group is enabled the
queue is never used (mutually exclusive modes).

- SmoothWrrState: nginx smooth weighted round-robin; ties keep the
  earlier member, stale member state is garbage-collected per pick
- select_route_providers resolves members against stored providers,
  drops disabled and dangling members, applies the same circuit-breaker
  filter as the failover branch, picks the WRR winner, and orders the
  rest in stored fallback order for in-request failover
- group doc is re-read per request so edits are hot
- unit tests: canonical 3:1 cycle, exact 30/10 over 40 picks, equal
  weights, single member, GC, mid-stream weight change; router tests
  for queue exclusion, disabled/dangling drops, breaker-open error,
  empty group error, winner-first ordering

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(proxy): per-member route model mapping
87cd692 
Exact request-model to actual-model mapping per route member, applied
before the existing role-keyword mappers so the mapped name inherits
normal [1m] handling and upstream override logic.

- apply_route_model_mapping rewrites body.model only on an exact table
  hit; unmatched models keep existing behavior
- RequestForwarder gains an optional route_model_mappings table
  (provider_id -> request -> actual) via with_route_model_mappings;
  absent mappings are a no-op
- request_builder applies the member table immediately before
  apply_model_mapping
- tests: rewrite-only-matched, empty-table no-op, route mapping
  feeding the role-keyword mapper

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(proxy): engage health checks and sync guard in route mode
4600402 
HandlerContext now loads route mode state per request: the active flag
plus the member model-mapping table, copied into both forwarder
construction sites.

- health_checks_engaged() = auto failover or route mode; replaces the
  failover-only predicate for bypass_circuit_breaker and the streaming/
  non-streaming timeout getters so route-mode apps get breakers and
  hung-member timeouts
- sync_successful_provider_selection returns early under route mode:
  no current-provider rewrite, no failover snapshot refresh, no
  failover_count bump; in-memory observability target is still recorded
- test: sync no-op assertions under route mode

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(proxy): route-mode lifecycle and exclusivity guards
f6d45b0 
Service-level lifecycle for route mode, restricted to the takeover
apps (claude/codex/gemini) like the failover flows it mirrors.

- enable_route_mode_for_app: requires at least one enabled member and
  no active automatic failover, reuses the failover always-on
  primitive for proxy routing, flips the flag last
- enable_proxy_and_route_mode_for_app: attaches the daemon-backed
  managed session and enables route mode in one step
- disable_route_mode_for_app flips the flag only; member config and
  proxy takeover stay untouched
- exclusivity: enable_auto_failover_for_app and the combined
  proxy+failover activation refuse while route mode is active, before
  any live-config mutation; the managed-session disable path refuses
  while route mode is active (startup recovery bypasses it)
- proxy show reports route mode with the enabled member count
- tests: every guard refusal, unsupported-app rejection, flag-only
  disable

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(tui): route-mode snapshot
9840f1d 
ProxySnapshot gains route_mode_enabled plus route_members projections
(provider id and display name, weight, enabled, mapping count) read
from the per-app route group doc on every snapshot load, so the Route
Mode screen reflects live edits. Dangling members fall back to their
raw provider id as the display name.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(tui): route mode screen
91a48c4 
New Route Mode screen as a top-level nav item for the proxy-capable
apps (Claude/Codex/Gemini), right after Providers.

- Screen: status row toggles route mode with Enter, member table shows
  name / enabled / weight / mapping count in stored fallback order,
  read-only endpoint footer; all long-form guidance lives in ? help
- Keys: w edits the member weight (1-1000), m edits the exact-match
  model-mapping JSON, a opens the member picker, d removes a member
- Member picker overlay lists all app providers with group membership
- Enable flow mirrors automatic failover: warns without an enabled
  member, and confirms ProxyEnableAndRouteMode when the app is not
  routed yet
- Runtime actions persist through the validating DAO so per-member
  model mappings survive edits; exclusivity and managed-session
  guards surface as error toasts
- ProxySnapshot members now carry the full mapping table for editor
  prefill
- Tests: nav order, intent-to-action coverage, runtime-action DAO
  effects (add/weight/toggle/mappings/remove + unknown-provider
  rejection), render assertions

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
test(proxy): route mode integration coverage
0f54e8f 
End-to-end tests over a live proxy with one mock upstream per route
member:

- 3:1 weights over 40 requests hit upstreams exactly 30/10
- mapped member receives the rewritten model, unmapped member receives
  the request model verbatim
- upstream 500 fails over in-request to the backup and the breaker
  records the failure
- failover-queued non-member receives zero requests while route mode
  is on
- lifecycle guards surface their text errors (empty group, unsupported
  app, failover conflict, failover-enable refusal, proxy-disable
  refusal)
- stale-takeover startup recovery keeps the route mode flag and member
  config
- hung member times out under non_streaming_timeout and the backup
  completes the request

Also exports RouteModeGroup/RouteModeMember from the library root for
integration tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
fix(proxy): route mode review fixes
1a23c85 
Blind-review findings, verified and fixed:

- set_route_mode_enabled no longer re-checks provider existence: a
  member referencing a deleted provider could previously wedge route
  mode on (disable refused, and proxy disable refused in turn). The
  flag flip now skips the existence check; direct member edits still
  validate. Unit test covers disable/re-enable over a dangling member.
- enabling route mode no longer reports automatic-failover wording
  when the daemon routing precondition fails: the shared precondition
  takes a feature label, so errors say what the user asked for.
- weight input pre-validates the 1..=1000 range so the toast matches
  the DAO rule instead of a differently-worded save error.
- removed the dead always-true wrapper around the route-mode guard in
  the managed-session disable path.
- daemon DropTakeover handler carries a comment pinning why the
  route-mode guard lives in the foreground caller only.
- route mode ? help documents that model mappings rewrite the request
  body's model field: they apply to Claude/Codex traffic while Gemini
  passthrough carries the model in the URL path (same scope as the
  existing role-keyword mapping).
- new tests: structurally-invalid object fails closed (read path),
  e2e proves a live group edit is visible to the very next request.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
fix(proxy): route mode second-review fixes
d4bc09b 
Round-two blind review follow-ups:

- member edits no longer fail because of a different, externally
  deleted member: the provider-existence check exempts members carried
  over unchanged from the stored doc (new or changed ids are still
  validated), so removing p2 no longer errors on dangling p1
- corrupt-doc warnings log once per app per process instead of at
  request rate (the request path re-reads the group 3-4x per request)
- dormant state is visible on primary surfaces: the Route Mode screen
  shows 'enabled (proxy stopped)' and proxy status shows 'route mode
  on (N members; proxy stopped)' when the flag is on but the proxy
  isn't running
- new unit test pins the route-mode wording of the daemon-routing
  refusal

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
fix(proxy): route mode third-review polish
118a3db 
- member mutations warn when a saved group is enabled but has no
  enabled member left, so the 'route mode on, zero members, all
  traffic failing' state is visible at the keypress that creates it
- the Route Mode screen's dormant qualifier now keys off per-app
  routing (routes_current_app_through_proxy) instead of the global
  proxy running flag, matching the codex-worker-only scenario
- new e2e: failover between two members with different mapping tables
  proves each attempt derives the body from the original request with
  its own member's mappings (no chaining across attempts)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
fix(proxy): route mode fourth-review fixes
988e70b 
- streaming e2e: a streaming request with a hung winner times out on
  the first byte and completes as SSE from the backup, covering the
  changed streaming timeout/breaker gates
- Codex e2e: route mode on the codex app rewrites body.model via the
  member mapping and passes unmapped models through verbatim
- the proxy-disable guard message no longer implies the proxy is
  currently running ('disable route mode first')
- proxy status pluralizes members and labels the section
  'Auto failover / route mode' now that the section shows both
- router unit test pins route-branch priority when both modes are on

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(tui): interactive route-mode editor with targeted refresh
24dda40 
Replace the keyboard-dead, freezing route-mode screen with a proper
interactive flow:

- Fix cursor navigation: Up and Down move independently (j/k via the
  vim normalizer); weight/mappings/delete bindings appear once a member
  row is selected.
- The member picker closes after add/remove and the cursor lands on the
  newly added row.
- Weight editing (w) validates 1..=1000 input and short-circuits
  equal-weight resubmission without a DB write or reload-token bump.
- Multi-row mappings editor (m): add, edit, and delete multiple
  request->actual pairs with duplicate/empty rejection (1-based row
  toasts), an empty-table clear, and Esc applying the whole table as
  one SetRouteModeMemberMappings action.
- Upstream model fetch (Ctrl+F) on the actual-model field supports
  Anthropic, Gemini-native, OpenAI-compatible Bearer, and Codex OAuth
  providers through a target-tagged model-fetch pipeline
  (ModelFetchTarget), keeping the existing form-field flow
  behavior-preserving; failures keep the editor and its draft.
- Disabling or removing the last enabled member is blocked with a
  warning toast while route mode is ON; free when OFF.
- Member mutations take the fast path: Database::init + targeted
  in-place refresh of route-mode data with mark-before-refresh and a
  best-effort targeted read, instead of a synchronous full UiData
  load. Reload-token staleness machinery discards stale full/snapshot
  loads, retargets pending warm loads on app switch, and drops stale
  snapshot refreshes so background reloads cannot revert visible
  edits.
- New i18n strings, contextual ? help entries, render cue for
  no-enabled-member groups, and 60+ tests across keymap, picker,
  editor, fetch, staleness, DAO persistence, and proxy integration.

Blind-reviewed through 13 rounds per the repository protocol; final
round reports all 9 acceptance criteria satisfied with only accepted
nit-level findings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(tui): route-mode request-model picker and passthrough seeding
cd04f9c 
Add Ctrl+R on the mapping editor's request field to open the model-fetch
picker pre-filled locally with request models read from the route app's
active provider config (Claude env model keys with [1m] markers stripped,
Codex top-level TOML model, Gemini GEMINI_MODEL). No network involved.

Adding a route member now seeds passthrough mappings (candidate ->
candidate) from the same pool, insert-if-absent only so existing keys are
never clobbered. Empty candidate pools keep the editor open with a toast.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
feat(tui): split route help per context with key bullets
cb6fc51 
Route mode's help was one prose blob shared by the route screen and the
model-mappings editor, and it never mentioned the new Ctrl+R/Ctrl+F row
keys or the passthrough auto-seed. Give the editor its own HelpTarget
that carries the provider name into the title, keep the member picker
on the screen help, and rewrite both blobs in the `· key — purpose`
bullet style used by the Codex catalog help (EN " — ", ZH "——").

Screen help keeps the rules (last enabled member, failover mutual
exclusion, proxy takeover, dormant persistence) and drops the nginx
analogy. Editor help covers mapping semantics, the Gemini URL-path
caveat, Esc apply-all with the stays-open-on-error report, and an
indented row-editing section. Help wording is aligned one-way to the
keymap chip vocabulary (delete member / add member / apply); shared
t! key labels are unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Hannibal @claude
fix(proxy): route mode fifth-review fixes
5e5c450 
- Route-model mapping: exact hit keeps priority; on miss retry the
  [1m]-stripped name so picker-seeded bare keys match suffixed requests
- Route-mode flag flip mutates the stored doc raw so structurally
  invalid members survive instead of being erased by a default-collapse
  rewrite
- Remove unused route_mode keymap help_items (help.rs uses a static
  block); add pinning tests for the mapping fallback, the raw flag
  flip, and the route-mode enable success path with a live managed
  worker

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@AtaraxiaEpiphany
AtaraxiaEpiphany marked this pull request as ready for review October 17, 2026 06:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@AtaraxiaEpiphany