[nachiketb-nvidia]

What and Why

Replace the binary relative-advantage implementation with one ordered routing policy. A rule names one candidate and its cutoff. The first matching rule selects that candidate; otherwise the explicit fallback is selected. The same evaluator handles two-model and N-way routing.

Review and merge order: #1013, then #1014.

Scope

This PR contains the libsy implementation, public API changes, and the existing consumer/test updates required for those changes to build and run:

• Required DecisionJudgeConfig.policy, with DecisionPolicy and candidate-only DecisionRule records.
• One candidate-choice request and one evaluator. Remove the relative-advantage prompt and branch.
• Minimal runner and PyO3 constructor/schema migrations, Python signatures, and existing Rust/Python/TypeScript fixtures.
• Stage/composite policies may select only their configured tier pair.

Deployment TOMLs, the Jev benchmark profile, server documentation, and published guides are deliberately in #1014. Standalone N-way TOML model-pool registration is also in #1014. Keeping the necessary constructor and fixture migrations here avoids a first PR that breaks compilation or its existing tests. No temporary legacy cutoff adapter is added.

Breaking Changes

• Rust policy, TOML decision.policy, and Python's policy argument are required. The old top-level cutoff is removed; thresholds live inside rules.
• Each rule has candidate and cutoff. There are no separate score and destination labels.
• The judge scores candidate labels instead of advantage and no_advantage. Custom instructions must use the candidate labels. Re-evaluate previous thresholds: candidate-choice and relative-advantage scores have different meanings.
• Missing or invalid verdicts select the policy fallback. Provider errors select it when fail_open is true, or propagate when false. In composite routing, the explicit fallback replaces the former implicit tier-retention behavior for unavailable decision verdicts.

Ordinary LLM judges and the general Decision protocol are unchanged. Merge #1014 next to migrate the shipped deployment examples.

Ordered N-way Selection

use libsy::{DecisionPolicy, DecisionRule};

let policy = DecisionPolicy {
    rules: vec![
        DecisionRule {
            candidate: "premium".into(),
            cutoff: 0.6,
        },
        DecisionRule {
            candidate: "balanced".into(),
            cutoff: 0.35,
        },
    ],
    fallback: "fast".into(),
};

Set this on DecisionJudgeConfig.policy, map the three labels to completion models in candidates, and register them in the runtime any group alongside the judge group.

if score[premium] > 0.60: select premium
else if score[balanced] > 0.35: select balanced
else: select fast

Premium	Balanced	Fast	Selected	Reason
0.65	0.25	0.10	premium	First rule matches.
0.45	0.40	0.15	balanced	Second rule matches, despite premium having the largest score.
0.60	0.36	0.04	balanced	Equality does not match the first rule.
0.60	0.35	0.05	fast	Both scores equal their cutoffs.
0.61	0.36	0.03	premium	Both rules match; declaration order wins.
0.30	0.30	0.40	fast	No rule matches.

All scores come from one judge call. The provider's selected option and confidence do not override the policy. Missing or invalid candidate scores use the fallback. Choice scores are not calibrated task-success probabilities.

What to Review

• decision.rs: policy validation, candidate selection, explicit fallback, and failure handling.
• decision_judge_routes_on_ordered_policy: existing binary/N-way coverage, strict cutoffs, invalid inputs, provider failures, and session affinity.
• Required consumer migrations: updated constructors and existing fixtures, without new routing behavior outside libsy.
• The PR has no deployment-profile or published-documentation changes; those are in feat(runner): expose N-way decision policies in configs and docs #1014.

Validation

• This PR independently passes the default-feature Rust workspace tests and workspace Clippy with all targets and warnings denied.
• Native TypeScript binding build, typecheck, and all three existing integration tests pass against loopback mocks.
• The complete stack also covers the existing Python decision/cancellation tests and the policy-only routing/server smoke checks.

No new test functions/files or dependencies. No paid provider calls. CI is not being awaited.

[github-actions]

PR Preview Action v1.8.1
🚀 View preview at https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-1013/
Built to branch gh-pages at 2026-10-19 15:57 UTC. Preview will be ready when the GitHub Pages deployment is complete.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Switchyard/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 1d99d9b4-a887-4a74-a7fb-82c72fc6b286

📥 Commits

Reviewing files that changed from the base of the PR and between 4cbb2d1 and 8482fdf.

📒 Files selected for processing: 6

• crates/libsy-llm-client/src/run.rs
• crates/libsy/src/algorithms/llm_class.rs
• crates/libsy/src/algorithms/llm_class/decision.rs
• crates/libsy/src/lib.rs
• crates/switchyard-py/src/libsy_bindings.rs
• crates/switchyard-runner/src/algorithm.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 10 remain after this review.

Walkthrough

Decision judges now support an optional ordered-cutoff policy with candidate score rules and a fallback target. The existing relative-advantage routing remains available. Public exports, configuration constructors, routing behavior, and tests are updated.

Changes

Decision Judge Routing

Layer / File(s)	Summary
Policy contract and configuration crates/libsy/src/algorithms/llm_class/decision.rs, crates/libsy/src/algorithms/llm_class.rs, crates/libsy/src/lib.rs, crates/switchyard-py/src/libsy_bindings.rs, crates/switchyard-runner/src/algorithm.rs, crates/libsy-llm-client/src/run.rs	Adds and exports DecisionPolicy and DecisionRule, adds optional policy configuration, and validates policy rules and labels. Existing configuration constructors set the policy to None.
Judge requests and routing crates/libsy/src/algorithms/llm_class/decision.rs, crates/libsy/src/algorithms/llm_class.rs	Policy mode sends candidate labels as judge choices. Routing validates probabilities, selects the first rule whose score exceeds its cutoff, and uses the fallback when no rule matches. Tests cover routing cases, invalid verdicts, and invalid policy configurations. Relative-advantage routing remains supported.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Merge Risk: ⚪ Minimal · up to 8482f

No actionable routing issue remains; the change is ready to merge after normal checks.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 33.33% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 6 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Title check		The title clearly describes the main change: routing decision choices through explicit policies. It is concise and related to the ordered-policy decision routing implemented in the pull request.
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.

• Fix all pre-merge checks with AI

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{I’m a rabbit with rules in a row,
I follow each score as I go.
If no cutoff is met,
The fallback is set,
And old paths still know where to go.}

_{Comment @coderabbitai help to get the list of available commands.}