Skip to content

feat(runner): support embedded Rust hosts - #912

Draft
afourniernv wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
afourniernv:codex/litellm-runtime-models
Draft

afourniernv wants to merge 2 commits into
NVIDIA-NeMo:mainfrom
afourniernv:codex/litellm-runtime-models

Conversation

@afourniernv

Copy link
Copy Markdown
Contributor

What

Adds AlgorithmSpec::build_with_runtime_models(...), which builds an algorithm and its matching runtime model groups from the same configuration.

Adds LlmClientError::Host so an embedded Rust host can preserve its native error without treating it as an upstream or FFI failure. Host error details are kept out of Switchyard telemetry.

Why

Embedded gateways could already build an AlgorithmSpec, but the runtime model-group mapping was private. A host such as LiteLLM would otherwise need to duplicate Switchyard's algorithm-specific model mapping.

The existing string and FFI error variants also could not preserve a typed native host error with the right semantics. These two additive surfaces let LiteLLM run Switchyard routing while continuing to own provider execution.

Notes for reviewers

Start with crates/switchyard-runner/src/algorithm.rs, then crates/protocol/src/client.rs.

No existing public methods or signatures are changed.

Validated with:

  • cargo fmt --all --check
  • cargo clippy --workspace --all-targets -- -D warnings
  • cargo test --workspace

Signed-off-by: Alex Fournier <afournier@nvidia.com>
Signed-off-by: Alex Fournier <afournier@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant