chore: single global router in front of engines for all deployments

[mikasenghaas]

Summary

Unifies inference deployment so that every shape — single-node, multi-node, and P/D-disaggregated — runs a single global router as the client-facing endpoint in front of the vLLM engines (first step towards a single-URL client, see RES-1156, RES-1028):

• Single node (local + SLURM): inference_local now spawns a vllm-router on server.port (default 8000) fronting the engine on backend_port (default 8100). The router's lifecycle is tied to the inference process (torn down on exit; a router crash tears down the engine via a watchdog thread). The RL entrypoint auto-wires orchestrator.model.client.admin_base_url to the engine port so admin ops (pause/update_weights/resume) bypass the router, matching multi-node semantics.
• Multi-node + P/D (SLURM): the per-replica routers are collapsed into one global router on inference node 0 fronting all per-rank endpoints across replicas (both vllm-router and llm-d). INFER_URLS is now a single URL; ADMIN_URLS still enumerate every rank directly. llm-d endpoint discovery now enumerates all replicas (prefill nodes across replicas first, then decode nodes).
• Config: router and backend_port move from the deployment configs to InferenceConfig top level — they describe the same client-facing layout in every deployment. The router listens on server.port (the per-router port field is gone). router = "None" runs a bare engine; the SLURM launchers write that into per-rank engine configs.
• Readiness: StaticInferencePool.wait_for_ready now health-checks the client-facing (router) endpoint in addition to the admin (engine) endpoints — the router only starts serving once its workers pass its own health check, so checking engines alone let rollouts race ahead of the router and fail with connection errors.

Breaking

• [inference.deployment.router] → [inference.router] (standalone entrypoint: [deployment.router] → [router]).
• inference.deployment.backend_port → inference.backend_port.
• inference.deployment.router.port is removed — the router always binds inference.server.port.
• Multi-node / P/D runs now expose one router URL instead of one per inference replica; the orchestrator receives a single --model.client.base-url.

Verification

On a 2-GPU node:

• uv run inference --model.name PrimeIntellect/Qwen3-0.6B-Reverse-Text-SFT: router comes up on :8000 fronting the engine on :8100; /v1/models, /v1/chat/completions, and /inference/v1/generate all served through the router; SIGTERM tears down both processes and frees the GPUs.
• uv run rl @ configs/basic/reverse-text/rl.toml --max-steps 3 --weight-broadcast.type filesystem (1 trainer + 1 inference GPU): subconfigs get base_url = :8000 (router) and admin_base_url = :8100 (engine); 3 steps complete with 0% rollout errors through the router and weight updates via the admin URL; the launcher exits 0 and the router process is gone afterwards. SIGTERM mid-run also tears down the full tree (router included).
• Rendered all five SLURM template shapes (single-node, multi-node, multi-node llm-d, disaggregated, RL disaggregated llm-d with 2 outer replicas) and bash -n checked them; llm-d endpoint enumeration across outer replicas maps logical→physical nodes correctly.
• pytest tests/unit (389 passed).

Note: with the default NCCL weight broadcast, the same 3-step run wedges at teardown — the trainer waits forever for broadcasts/step_1/NCCL_READY after the orchestrator has exited. This reproduces identically on main (pre-existing tail race, the known NCCL_READY timeout follow-up from RES-1077) and is unrelated to this PR; the CI reverse-text config already pins weight_broadcast.type = "filesystem" for the same reason.

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches core inference launch paths, SLURM templates, and orchestrator client URLs; breaking config renames and a single router URL for multi-replica runs require config and client updates.

Overview
Every deployment shape now exposes one client URL on inference.server.port, with vLLM engines on inference.backend_port (+ rank offset). Config moves and breaking renames: [inference.deployment.router] → [inference.router], backend_port to top-level InferenceConfig, and per-router port removed (always server.port).

Local / single-node: inference_local spawns vllm-router on server.port and binds the engine to backend_port, with lifecycle tied to the inference process. RL auto-sets orchestrator.model.client.admin_base_url to the engine port so weight updates bypass the router.

Multi-node / P/D SLURM: Per-replica routers are replaced by one global router on inference node 0; INFER_URLS is a single URL while ADMIN_URLS still list every rank. Per-rank engine TOMLs set router = None; the sbatch script starts the router. llm-d endpoint discovery enumerates all replicas’ prefill then decode nodes.

Readiness: StaticInferencePool.wait_for_ready also health-checks the router when admin_base_url is set, avoiding rollouts before the router is up.

Validators require a router for multi-node/disaggregated deployments and restrict llm-d to SLURM multi-node/disaggregated (not single-node local).

^{Reviewed by Cursor Bugbot for commit 7f39bec. Bugbot is set up for automated code reviews on this repo. Configure here.}