[hakankaan]

Summary

Adds ModelRunner as a multi-model provider gateway: one MODELRUNNER_KEY and one async queue contract in front of 17 verified routes across all four paid generation capabilities — video, image, TTS, and music. This is the implementation proposed in #569 and follows the accepted Atlas Cloud gateway shape: a shared client module, an explicit verified route catalog, and one thin BaseTool per capability, discovered by the existing selectors with zero selector code changes.

Disclosure: I work at ModelRunner. I'm contributing and committing to maintain this integration, and the testing evidence below includes live billed requests against production endpoints.

Related issue

Closes #569

Why ModelRunner belongs (provider viability)

• Coverage a single key doesn't have today: music currently has only three cloud providers and TTS has no gateway-style option. This PR adds one key covering video (Wan 2.7, Happy Horse 1.1 with lip-synced dialogue, Seedance 2.0 Mini budget tier), image (Seedream 5.0/Pro, Recraft V4.1/Pro, SD 3.5 Large), TTS (Kokoro, Gemini Flash TTS, ElevenLabs Multilingual v2, Chatterbox), and music (Lyria 2, Lyria 3 Clip, Stable Audio 2.5).
• Public, testable, machine-readable: the catalog at https://modelrunner.ai/models publishes per-model OpenAPI schemas and pricing, so contributors can verify routes and rates without an account.
• Clear pricing for cost governance: every route in tools/modelrunner_models.py carries an explicit pricing mode and rate table (including resolution-tiered per-second video rates), snapshot-dated (2026-08-28) with source comments, feeding truthful estimate_cost() before any paid call.
• Maintenance commitment: as the affiliated contributor I'll keep routes, rates, and the queue contract current, and I have live-tested access for regression smoke.

Changes

Provider gateway (new)

• tools/modelrunner_client.py — shared queue client: submit → poll status_url → fetch response_url; monotonic poll deadline (sleeps and per-request timeouts capped to the remaining budget), bounded transport-error tolerance, result-envelope normalization (scalar vs list outputs per route), storage upload (streamed from disk) for image-to-video. Auth boundary: the key is attached only to the two first-party hosts, authenticated calls never follow redirects, output-download URLs are validated (https, public host) and re-validated on every redirect hop, and upload destinations from the initiate response are held to the same bar. A submission whose queue envelope turns out unusable still surfaces the request_id for billing provenance. requests is imported lazily so registry discovery never depends on it.
• tools/modelrunner_models.py — verified route catalog: 17 routes (5 video, 5 image, 4 TTS, 3 music) with per-route pricing modes, configuration-keyed rate tables, output specs, and capability flags.
• tools/video/modelrunner_video.py, tools/graphics/modelrunner_image.py, tools/audio/modelrunner_tts.py, tools/audio/modelrunner_music.py — one BaseTool per capability (provider="modelrunner", stability=beta). Fail-before-billing validation: extra_params cannot override validated fields, poll controls are parsed before any paid submit, payload validation runs before the storage upload, and an explicitly pinned endpoint is never silently rewritten to a sibling route. Post-submit failures return cost_usd=0.0 with request_id/status_url/response_url and billing_status="possibly_billed"; nothing is ever auto-resubmitted. The video tool deliberately does not advertise image_url in its schema (the seedance_ark convention), so video_selector hands local reference images through for ModelRunner-storage hosting instead of fal-uploading them — image-to-video works on a machine with only MODELRUNNER_KEY.

Skill & docs

• .agents/skills/modelrunner/SKILL.md — Layer-3 skill: route/price tables, canonical execute example, result and failure contract, troubleshooting.
• docs/PROVIDERS.md, docs/ARCHITECTURE.md, AGENT_GUIDE.md, skills/INDEX.md, README.md, README_zh-CN.md, PROJECT_CONTEXT.md, .env.example — provider registration, env keys (MODELRUNNER_KEY, plus MODELRUNNER_API_KEY/MRUN_API_KEY aliases), route/price tables.

Pipeline exposure

• Seven music-bearing pipeline manifests + matching asset-director skills now list modelrunner_music (manifests are the authoritative tool lists; video/image/TTS flow through selectors automatically and need no manifest edits).

Tests (per docs/PR_REVIEW_GUIDE.md minimum coverage for a new provider — all mocked, no sockets)

• tests/contracts/test_modelrunner_tools.py — identity/metadata contracts, 3-alias status checks, no-key fails with cost_usd == 0.0 before any socket, registry discovery, selector exact-model narrowing, the schema-must-not-advertise-image_url selector contract, payload builders per route, lazy-import guarantee (AST-checked), pricing-calculator branches incl. resolution tiers.
• tests/tools/test_modelrunner_video.py — happy path per modality, FAILED / COMPLETED-with-error, poll timeout with possibly_billed data, transport-error tolerance, image-to-video upload flow, auth boundary (no Authorization off-host, redirects on authenticated calls refused, download redirect hops re-validated, unusable queue envelopes still carry request_id), and the fail-before-billing guards above.
• tests/tools/test_video_selector_routing.py — the selector passes a local reference image through to modelrunner_video untouched (no FAL upload), using the real tool schema so drift breaks the test.
• tests/test_network_guard.py — ModelRunner fails-instead-of-billing case.
• tests/contracts/test_phase3_contracts.py — registry TTS provider set extended with modelrunner.

Testing

• make lint && make test: 1947 passed, 12 skipped, 3 xfailed locally (macOS, Python 3.10 venv; syntax is 3.10-compatible).
• Registry smoke: all four tools discovered; with only MODELRUNNER_KEY set, each capability menu reports modelrunner configured (video 1/27, image 1/17, TTS 1/11, music 1/6).
• Selector smoke: video_selector rank mode includes modelrunner_video; an exact-model pin (wan-video/wan/v2.7/text-to-video) narrows the ranking to ['modelrunner_video']; tts_selector routes preferred_provider="modelrunner" + model_id to the tool. No selector code was changed.
• Live paid smoke (scripts/modelrunner_media_smoke.py, opt-in --allow-paid, routed through the selectors so it also proves discovery and routing). Billed prices read back from ModelRunner's request records:

Case	Endpoint	Request ID	Estimated	Billed	Output (ffprobe)
TTS narration (via tts_selector)	hexgrad/kokoro-82m	VJchT3Z20ZRxc2kJz4Z2G	$0.000102	$0.000073	7.9s WAV
Image still (via image_selector)	bytedance/seedream-v5/text-to-image	CykylRXcvtbblKDUk0iqA	$0.035	$0.035	2048×2048 JPEG
Music bed	google/lyria-3/clip	t6r427XUjLdAO1yogfcQW	$0.04	$0.04	30.8s MP3
Budget video (via video_selector)	bytedance/seedance-v2-mini/text-to-video	376rdaTOCPXtsKgVVdurw	$0.212	$0.2173	4.1s 864×496 h264 + audio
Flagship video (via video_selector)	wan-video/wan/v2.7/text-to-video	TinqrOx4RlmgaLJS8Huzz	$0.40	$0.40	4.04s 1280×720 h264 + audio

Three exact estimate↔billed matches; Kokoro billed below its upper-bound estimate; Seedance billed on its delivered 4.1s output (per-second routes bill delivered length — now noted in the skill).

• Live negative-path evidence: during the smoke window, google/lyria2 happened to be failing upstream. The gateway surfaced each failure as a failed ToolResult with cost_usd=0.0, and ModelRunner's request records confirm billingStatus: failed / $0 for all three attempts (nR9nDLH2r88QNwPndrhJE, 3fUgghHkyeVvPrwjge2Hf, KZ2gMsILqMYiJwzFtoTFN) — a real-world exercise of the fail-honestly contract, including the COMPLETED-with-error envelope the client is built to reject.

Preserved deliberately

• No selector routing changes — all four tools are picked up by existing capability discovery, and the video tool's schema follows the existing seedance_ark convention for self-hosted local references.
• Conservative supports flags — tools only advertise what every default route guarantees; route-specific extras (lip-sync, native-audio toggles) live in model_catalog metadata so ranking isn't inflated.
• stability=beta and no quality_score self-promotion for an affiliated provider.
• Catalog rates are a dated snapshot; the skill instructs agents to verify pricing against the live catalog before large batches.

If you'd prefer a smaller first landing, I'm happy to split this into core+video/image first with TTS/music as a follow-up.

Checklist

• The change is focused on a single logical concern.
• I ran the relevant tests locally (make test-contracts / make test) where applicable.
• I updated docs/README if behavior or usage changed.
• No unrelated files (build artifacts, local config) are included in the diff.

🤖 Generated with Claude Code

[calesthio]

Thank you for the documented ModelRunner catalog, validation and auth boundaries. The current public catalog is active, and all 107 focused contract/transport tests pass. The billing/recovery contract needs correction before approval.

[P1] Preserve unknown charges after accepted or ambiguous submissions. All four _failure methods set cost_usd=0.0 even when queue data says billing_status=possibly_billed; a lost POST response also falls into the zero-cost path. An error after acceptance or download failure can still represent a completed billable generation. Return None for unknown actual charges, retain all known request coordinates, and explicitly mark ambiguous submit outcomes as unsafe to resubmit. A billing note alongside zero does not fix the ledger value.

[P1] Add collection/resume without a second paid POST. Each execute method always submits; there is no queue/resume input that consumes the status_url/response_url returned on failure. The recovery information is therefore insufficient for the agent to finish the same job through the tool. Add a validated GET-only collect path (including artifact download recovery), persist accepted queue coordinates, and test that rerunning collection does not submit.

[P2] Reconcile delivered video usage rather than requested seconds. ModelRunnerVideo returns estimate_cost using payload duration even though the PR's own billed example shows a requested 4s Seedance clip billed at delivered 4.1s. Use verified provider billing/usage, or preserve an explicitly estimated/unknown charge; also reserve enough to cover documented output rounding.

No live or paid request was made.

[hakankaan]

Thanks. All three points were right. Here's how this push addresses each one.

P1a: a failure reported cost_usd=0.0 for a job that may have billed.

• cost_usd is now the charge read back from the request record (GET /requests/{id} → totalPrice / billingStatus), not the estimate. When that charge can't be verified it is None, never 0.0.

• Every failure carries billing_status and resubmit_safe:

  billing_status	cost_usd	resubmit_safe
  not_submitted	0.0	true
  unknown	None	false
  possibly_billed	None	false
  failed	0.0	true
  charged / partial	the charge	false

• The lost-response case had no fix on our side either, so we added one to the API: POST /{owner}/{alias} now accepts X-Idempotency-Key. It is deployed. The client sends a per-attempt key and re-sends lost, 5xx and 429 answers under that same key, so a retry returns the original job instead of starting a second paid one.

• The client never reads a refusal's status code as proof that nothing dispatched. It re-sends once under the same key, and the server answers with the same refusal only if the key was released.

P1b: there was no way to collect a submitted job without resubmitting.
Pass request_id (or status_url / response_url) back to any of the four tools. The tool polls and downloads, and makes no POST. A collect call reports cost_usd=0.0 and puts the job's own charge in data["billing"]. Every post-submit failure includes a recovery_hint pointing at this.

P2: estimates were lower than the charge.
Per-second video and per-audio-second TTS estimates now add one second of headroom (rate × (seconds + 1)), citing the 4s → 4.1s Seedance case. Flat-rate routes are unchanged.

One core-type change that needs your OK: ToolResult.cost_usd is now Optional[float] (default still 0.0). CostTracker.reconcile(entry_id, None) counts the entry at its reservation, or its estimate if it was never reserved, and flags it actual_unknown. That keeps the budget conservative instead of booking an unknown charge as free. reconcile() has no production callers today, so nothing else moves. If you'd rather keep cost_usd: float, the alternative is to report the estimate plus a flag, but then an unknown charge looks the same as a known one.

Tests extend the existing suites: tests/tools/test_modelrunner_video.py (billing read-back, idempotent re-send, collect) and tests/contracts/test_modelrunner_tools.py (headroom tables, metadata reserved). make test: 1963 passed, 11 skipped, 3 xfailed.

Live check against production (Kokoro TTS). The same X-Idempotency-Key sent twice returned 201 and then 200 with the same request_id, and a changed body under that key returned 409. A tool run reported cost_usd=0.000035, which matches the request record's totalPrice. Collecting that run by request_id downloaded the audio, reported cost_usd=0.0, and created no new request.

[hakankaan]

@calesthio all three points are addressed in 0167758 (details in the comment above). Could you take another look when you have a moment?

[hakankaan]

Status update, two commits since the reply above:

• bc24f68 closes a gap I left in P1b. You asked for accepted queue coordinates to be persisted; I had only returned them in the ToolResult, so a process that died while polling lost the request_id. Each submit now writes a job file through provider_jobs.save_job, at job_path (default <output_path>.job.json): the attempt id before the POST, then request_id, status_url and response_url once the job is accepted. An unwritable job_path fails before anything is sent. Three tests cover it in tests/tools/test_modelrunner_video.py.
• 7d1d973 merges current main, so the conflicts are gone.

Two things in my earlier reply no longer apply:

• main already has ToolResult.cost_usd: Optional[float], so there is no core-type change left for you to approve. The only core touch is CostTracker.reconcile accepting None.
• The test count is out of date. Locally on bc24f68: make lint passes, and the suite is 3034 passed, 17 skipped with tests/tools/test_remotion_caption_burn.py deselected. Two Thai caption tests in that file fail on my machine on a clean main as well.

CI on this PR is waiting for workflow approval, so it hasn't run yet.