feat: add ModelRunner multi-model gateway (video, image, TTS, music)

.agents/skills/modelrunner/SKILL.md

@@ -0,0 +1,177 @@
+---
+name: modelrunner
+description: Generate video, images, speech, or music through the ModelRunner gateway. Use for ModelRunner-hosted Wan 2.7, Happy Horse, Seedance 2.0 Mini, Seedream 5.0, Recraft V4.1, Stable Diffusion 3.5, Kokoro, Gemini TTS, ElevenLabs, Lyria, or Stable Audio, or when one MODELRUNNER_KEY should cover all four media capabilities.
+metadata:
+  openclaw:
+    requires:
+      env_any:
+        - MODELRUNNER_KEY
+        - MODELRUNNER_API_KEY
+        - MRUN_API_KEY
+---
+
+# ModelRunner
+
+Route complete productions through `video_selector`, `image_selector`, or
+`tts_selector`. Call `modelrunner_video`, `modelrunner_image`,
+`modelrunner_tts`, or `modelrunner_music` directly when the user names
+ModelRunner or an exact ModelRunner endpoint. Never substitute a different
+provider for a ModelRunner request.
+
+Set `MODELRUNNER_KEY` (get one at https://modelrunner.ai -> Settings -> API
+Keys). The aliases `MODELRUNNER_API_KEY` and `MRUN_API_KEY` are accepted.
+
+## Preflight every paid call
+
+1. Read `tool.get_info()["model_catalog"]`; do not infer a route from a model
+   family name or invent a task suffix — endpoint ids are exact.
+2. Several routes bill per second at a rate that DEPENDS ON `resolution`
+   (the tables below). Announce the tool, exact endpoint, request count, and
+   estimated cost before submitting. Per-second routes bill on the DELIVERED
+   clip length, which can run slightly past the requested whole seconds
+   (e.g. a 4s Seedance request delivering 4.1s bills ~2.5% over estimate).
+3. Local image paths are fine — the video tool uploads them through
+   ModelRunner storage first.
+4. Save outputs inside the active `projects/<project-id>/` tree and inspect them.
+
+## Video routes
+
+Every video route generates a synchronized soundtrack with the picture by
+default; the Seedance routes can opt out with `generate_audio: false`.
+
+| Route | Operation | Rate per output second | Notes |
+|---|---|---|---|
+| `wan-video/wan/v2.7/text-to-video` | text | $0.10 @720P / $0.15 @1080P | 2-15s; 16:9, 9:16, 1:1, 4:3, 3:4; tool defaults to 720P for drafting |
+| `wan-video/wan/v2.7/image-to-video` | image | $0.10 @720P / $0.15 @1080P | 2-15s; optional `end_image_url` closing frame; frame shape follows the source image (no ratio field) |
+| `alibaba/happy-horse/v1.1/text-to-video` | text | $0.14 @720P / $0.18 @1080P | 3-15s; dialogue written in the prompt is spoken and lip-synced; 9 ratios incl. 21:9 and 4:5 |
+| `bytedance/seedance-v2-mini/text-to-video` | text | $0.053 @480p / $0.113 @720p | 4-15s; budget drafting tier; `generate_audio: false` for silence (same price) |
+| `bytedance/seedance-v2-mini/image-to-video` | image | $0.053 @480p / $0.113 @720p | 4-15s; motion prompt required; `aspect_ratio: adaptive` keeps the source shape |
+
+```python
+tool.execute({
+    "prompt": "The balloon drifts across the valley as sunrise light spreads over the ridges",
+    "model": "wan-video/wan/v2.7/image-to-video",
+    "operation": "image_to_video",
+    "image_path": "projects/demo/frame.png",
+    "duration": 5,
+    "resolution": "720P",
+    "output_path": "projects/demo/balloon.mp4",
+})
+```
+
+## Image routes
+
+Image routes return a LIST of hosted URLs; extra images land as `stem_2`, ...
+
+| Route | Price | Sizing |
+|---|---|---|
+| `bytedance/seedream-v5/text-to-image` | $0.035 flat | `size` WxH enum, all 2K-class (default 2048x2048) |
+| `bytedance/seedream-v5-pro/text-to-image` | $0.045 (~1MP sizes) / $0.09 (2K sizes) | `size` WxH enum, 14 values — the price follows the size |
+| `recraft/v4.1/text-to-image` | $0.035 flat | `image_size` preset or `{width, height}` up to 2048 |
+| `recraft/v4.1/pro/text-to-image` | $0.21 flat | same sizing, premium tier |
+| `stability-ai/stable-diffusion-v3.5-large` | $0.065 per output megapixel | `image_size` preset or `{width, height}`; also takes `negative_prompt`, `guidance_scale`, `num_inference_steps` |
+
+Canonical `width`/`height` are adapted to the closest supported size; passing
+the model-native `size` / `image_size` value instead is validated strictly and
+fails before billing.
+
+## Speech routes (`modelrunner_tts`)
+
+| Route | Price | Voice field means |
+|---|---|---|
+| `hexgrad/kokoro-82m` (default) | GPU compute time — a fraction of a cent per clip | one of 46 named voices; prefix = language+gender (`af_bella`, `ff_siwis`, `jm_kumo`) |
+| `google/gemini-3.1-flash-tts` | $0.0006 per audio second | 30 prebuilt voices (Kore, Puck, ...); pass delivery direction via `instructions` |
+| `elevenlabs/tts/multilingual-v2` | $0.0015 per audio second | 21 named voices (Rachel, ...); `stability`/`similarity_boost`/`style` honored |
+| `resemble-ai/chatterbox/text-to-speech/multilingual` | $0.000375 per audio second | `voice` selects the LANGUAGE; 300-char cap per call |
+
+`model_id` accepts the friendly aliases `kokoro`, `gemini-tts`,
+`multilingual-v2`, and `chatterbox`. The tts_selector has no exact-model
+narrowing — reach a specific route with
+`preferred_provider="modelrunner"` plus `model_id`, or call the tool directly.
+Controls a route does not support are ignored, not rejected (the selector
+passes its full vocabulary through); the catalog's `controls` list says what
+each route honors.
+
+## Music routes (`modelrunner_music`)
+
+| Route | Price | Duration |
+|---|---|---|
+| `google/lyria2` (default) | $0.06 flat | fixed ~30s, instrumental only, 48kHz WAV |
+| `google/lyria-3/clip` | $0.04 flat | fixed ~30s, sings lyrics by default — prompt "instrumental only" to suppress |
+| `stability-ai/stable-audio-2.5/text-to-audio` | $0.20 flat | `duration_seconds` 1-190; the one route with duration control |
+
+A `duration_seconds` the route cannot honor fails before billing and names the
+route that can.
+
+## Result and failure contract
+
+All four tools submit to the ModelRunner queue, poll the returned request id,
+download the output, and return `ToolResult` provenance containing the
+provider, exact endpoint, request id, submitted parameters, source URL,
+artifact paths, and cost. An unknown route, invalid enum, missing media input,
+or out-of-range duration fails BEFORE billing, with the supported choices in
+the error. A request can report COMPLETED with a non-empty `error` (a
+normalized provider failure); the tools treat that as failure, never as output.
+
+**Cost.** After a job finishes, the tool reads the request record back from
+ModelRunner and reports what was actually charged. `data["billing"]` says
+where the figure came from:
+- `"source": "modelrunner_request_record"` means it is the real charge;
+- `"source": "estimate"` means the charge had not settled when the tool read
+  it back.
+
+`cost_usd` is `None` when the charge is **unknown**: the job was or may have
+been accepted, and its outcome cannot be verified. It is never `0.0` in that
+case.
+
+**Failures.** Every failure carries `billing_status` and `resubmit_safe`:
+
+| `billing_status` | Meaning | `cost_usd` | `resubmit_safe` |
+|---|---|---|---|
+| `not_submitted` | Refused before anything was accepted | `0.0` | true |
+| `not_collected` | A collect call failed before reaching the job | `0.0` | false |
+| `unknown` | The submit's outcome is unknown | `None` | false |
+| `possibly_billed` | Accepted; the charge has not settled | `None` | false |
+| `failed` | The job failed and was not charged | `0.0` | true |
+| `charged` / `partial` | Charged (e.g. the download failed after completion) | the charge | false |
+
+When `resubmit_safe` is false, do not resubmit. Collect the existing job
+instead: pass the failure's `request_id` (or its `status_url` /
+`response_url`) back to the same tool. A collect call polls, downloads and
+reports the job without submitting anything:
+
+```python
+tool.execute({"request_id": failed.data["request_id"], "output_path": "out.mp4"})
+```
+
+**Job checkpoint.** Every submission is also written to disk, at `job_path`
+(default `<output_path>.job.json`). The attempt id goes in before the request
+is sent, and the `request_id`, `status_url` and `response_url` once the job is
+accepted. If the process dies while polling, read the `request_id` from that
+file and collect the job. A file still in `"state": "submitting"` or
+`"unknown"` has only the `attempt_id`: find the job in the request history by
+its `openmontage_attempt` metadata before submitting again.
+
+A collect call reports `cost_usd: 0.0`, because it caused no new spend. The
+job's own charge is in `data["billing"]`.
+
+**Submits are idempotent.** Each submit sends a per-attempt
+`X-Idempotency-Key` and retries transport errors, 5xx and 429 responses under
+that same key, so a lost response never starts a second paid job. A refusal is
+confirmed by one more same-key send before it is reported as `not_submitted`:
+the server answers the same refusal only when nothing was dispatched. If the outcome
+is still unknown after the retries, the failure carries an `attempt_id`, and
+the request is tagged with metadata `openmontage_attempt=<attempt_id>`. Find
+it in your ModelRunner request history before resubmitting. `extra_params`
+may not set `metadata`.
+
+**Estimates.** Per-second video and per-audio-second TTS estimates add one
+second of headroom (`rate x (seconds + 1)`). Billing is on the seconds
+delivered, which can run past the request: a 4s Seedance clip delivered 4.1s.
+
+The API key is only ever sent to `queue.modelrunner.run` and
+`api.modelrunner.run`; output downloads are unauthenticated.
+
+Rates above were verified 2026-08-28. Confirm current pricing from
+`get_info()["model_catalog"]` and the live model page before quoting or
+spending on a batch.

.env.example

@@ -8,6 +8,15 @@ FAL_KEY=
 # Alias for FAL_KEY (some SDKs/docs use this name); either one is read.
 FAL_AI_API_KEY=
 
+# --- ModelRunner (one key: video + image + TTS + music) ---
+# Wan/Happy Horse/Seedance video, Seedream/Recraft/SD 3.5 images,
+# Kokoro/Gemini/ElevenLabs speech, Lyria/Stable Audio music.
+# Get one at https://modelrunner.ai -> Settings -> API Keys
+MODELRUNNER_KEY=
+# Aliases for MODELRUNNER_KEY (used in some ModelRunner docs); any one is read.
+# MODELRUNNER_API_KEY=
+# MRUN_API_KEY=
+
 # --- MiniMax official direct API ---
 # First-party image generation plus MiniMax H3 / older Hailuo video generation.
 # Get one at https://platform.minimax.io/user-center/basic-information/interface-key

AGENT_GUIDE.md

@@ -694,9 +694,9 @@ The `.agents/skills/` directory is large. When you're not coming in through a to
 | **Composition runtime** | `remotion`, `remotion-best-practices`, `synthetic-screen-recording` (fake terminal/UI demos via Remotion TerminalScene), `threejs-world-generation` (semantic terrain and free-viewpoint HyperFrames worlds) |
 | **Animation knowledge (generic)** | `gsap-core`, `gsap-timeline`, `gsap-plugins` (SplitText / MorphSVG / DrawSVG / MotionPath / Flip / CustomEase), `gsap-utils`, `gsap-react`, `gsap-performance`, `gsap-scrolltrigger`, `gsap-frameworks`, `framer-motion` (Disney 12 principles), `lottie-bodymovin` (Lottie export) |
 | **Character animation** | `character-rigging`, `svg-character-animation`, `pose-library-design`, `canvas-procedural-animation`, `character-animation-qa` |
-| **Image generation** | `bfl-api`, `flux-best-practices` |
-| **Video generation** | `seedance-2-0` (preferred premium default — cinematic, trailer, multi-shot, synced audio, lip-sync), `gemini-omni` (conversational video editing, reference tags, timecoded beats), `ai-video-gen`, `ltx2` |
-| **Audio** | `elevenlabs`, `music`, `sound-effects`, `acestep`, `text-to-speech`, `azure-text-to-speech` (optional cloud TTS — tool `azure_tts`, same Speech key as `azure_stt`), `setup-api-key` |
+| **Image generation** | `bfl-api`, `flux-best-practices`, `modelrunner` (Seedream, Recraft, SD 3.5 via one ModelRunner key) |
+| **Video generation** | `seedance-2-0` (preferred premium default — cinematic, trailer, multi-shot, synced audio, lip-sync), `gemini-omni` (conversational video editing, reference tags, timecoded beats), `ai-video-gen`, `ltx2`, `modelrunner` (Wan 2.7, Happy Horse, Seedance Mini via one ModelRunner key) |
+| **Audio** | `elevenlabs`, `music`, `sound-effects`, `acestep`, `text-to-speech`, `azure-text-to-speech` (optional cloud TTS — tool `azure_tts`, same Speech key as `azure_stt`), `modelrunner` (Kokoro/Gemini/ElevenLabs TTS + Lyria/Stable Audio music), `setup-api-key` |
 | **Speech-to-text** | `speech-to-text` (whisper `transcriber` — default, offline), `azure-speech-to-text` (optional cloud STT — tool `azure_stt`, preferred when `AZURE_SPEECH_KEY` is set) |
 | **Avatar / lip-sync** | `avatar-video`, `heygen`, `create-video`, `faceswap`, `video-translate`, `agents` |
 | **Capture** | `playwright-recording` (browser flows), `ffmpeg` (post) |

PROJECT_CONTEXT.md

@@ -75,6 +75,8 @@ Each tool's `agent_skills[]` field bridges Layer 1 → Layer 3. See `skills/INDE
 | `tools/graphics/threejs_world.py` | Local semantic 3D-world authoring with explicit blockout/production fidelity tiers, region-aware terrain, diagnostics, and HyperFrames atelier workspaces |
 | `tools/graphics/threejs_asset_catalog.py` | CC0 GLTF/GLB catalog acquisition, inventory, and provenance for production-fidelity world builds |
 | `tools/graphics/atlas_3d.py` | Atlas Cloud Tripo H3.1 text-to-3D for unique textured/PBR GLB assets |
+| `tools/modelrunner_client.py` | Shared ModelRunner queue/storage plumbing (submit, poll, result, upload, download) |
+| `tools/modelrunner_models.py` | Verified ModelRunner route catalog with per-route rate tables for video/image/TTS/music |
 | `tools/graphics/fal_3d.py` | fal.ai Hunyuan 3D and SAM 3D routes for image-conditioned and multi-object GLB generation |
 | `tools/graphics/blender_world.py` | Blender 4.5 LTS production world assembly, terrain, lighting, camera, and Eevee Next rendering |
 | `tools/character/character_animation.py` | Local character-animation tools — character specs, SVG rig plans, pose libraries, action timelines, HyperFrames packages, and QA reports |

README.md

@@ -234,6 +234,7 @@ This repo is built for agentic operation. If you're an OpenClaw-style agent, her
 # Image + video gateway:
 FAL_KEY=your-key               # FLUX images + Google Veo, Kling, MiniMax video + Recraft images
 ATLASCLOUD_API_KEY=your-key    # Atlas Cloud — Seedream/Nano Banana/GPT Image + Kling/Seedance/Hailuo video
+MODELRUNNER_KEY=your-key       # ModelRunner — Wan/Seedance video + Seedream/Recraft/SD3.5 images + Kokoro/Gemini TTS + Lyria/Stable Audio music
 
 # Kling official direct API:
 KLING_API_KEY=your-key         # Official Kling video, image, TTS, avatar, lip sync
@@ -493,6 +494,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
 | **Kling (fal.ai)** | Cloud API | High quality, fast via fal.ai gateway |
 | **Kling Official** | Cloud API | Official direct API with separate `kling_official` provider |
 | **Atlas Cloud** | Cloud API | Unified gateway for Seedance, MiniMax, Hunyuan, and other multimodal models |
+| **ModelRunner** | Cloud API | One-key gateway: Wan 2.7, Happy Horse (lip-synced dialogue), Seedance Mini budget tier |
 | **Seedance 2.0 (Volcengine Ark)** | Cloud API | Official direct API with separate `seedance_ark` provider |
 | **Seedance 2.5 / 2.0** | Cloud API | Text, image, and reference-driven video workflows through supported gateways |
 | **Gemini Omni Flash** | Cloud API | Conversational multimodal video generation and editing |
@@ -524,6 +526,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
 | **Seedream 5.0** | Cloud API | High-fidelity text-to-image and image editing through supported gateways |
 | **Nano Banana 2** | Cloud API | Multimodal image generation and editing |
 | **Atlas Cloud** | Cloud API | Unified access to multiple image-generation model families |
+| **ModelRunner** | Cloud API | Seedream 5.0 Lite/Pro, Recraft V4.1, Stable Diffusion 3.5 through one key |
 | **Recraft** | Cloud API | Design-focused generation |
 | **Kling Official** | Cloud API | Official direct API for Kling image generation and reference workflows |
 | **Local Diffusion** | Local GPU | Stable Diffusion, free |
@@ -546,6 +549,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
 | **Piper** | Local | Completely free, offline |
 | **Azure Speech** | Cloud API | Fast multilingual speech services |
 | **DashScope / Doubao / Fish Audio** | Cloud API | Additional multilingual and expressive voice options |
+| **ModelRunner TTS** | Cloud API | Kokoro (sub-cent), Gemini TTS (directable delivery), ElevenLabs, Chatterbox via one key |
 
 </details>
 
@@ -559,6 +563,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
 | **Suno AI** | Cloud API | Full song generation with vocals, lyrics, any genre. Up to 8 minutes. |
 | **ElevenLabs Music** | Cloud API | AI music generation |
 | **ElevenLabs SFX** | Cloud API | Sound effect generation |
+| **ModelRunner Music** | Cloud API | Lyria 2/3 clips and Stable Audio 2.5 long-form tracks via one key |
 
 **Post-Production (always available, always free):**
 

README_zh-CN.md

@@ -175,6 +175,7 @@ make setup
 
 # 图像 + 视频网关:
 FAL_KEY=your-key               # FLUX 图像 + Google Veo、Kling、MiniMax 视频 + Recraft 图像
+MODELRUNNER_KEY=your-key       # ModelRunner — Wan/Seedance 视频 + Seedream/Recraft/SD3.5 图像 + Kokoro/Gemini TTS + Lyria/Stable Audio 音乐
 
 # 免费素材库:
 PEXELS_API_KEY=your-key        # 免费库存视频和图像
@@ -422,7 +423,7 @@ OpenMontage/
 > **包含定价与免费额度的完整设置指南：** [`docs/PROVIDERS.md`](docs/PROVIDERS.md)
 
 <details>
-<summary><strong>视频生成 — 15 家提供商</strong></summary>
+<summary><strong>视频生成 — 16 家提供商</strong></summary>
 
 | 提供商 | 类型 | 备注 |
 |----------|------|-------|
@@ -434,6 +435,7 @@ OpenMontage/
 | **Higgsfield** | 云端 API | 带 Soul ID 以实现角色一致性的多模型编排器 |
 | **MiniMax** | 云端 API | 极具成本效益 |
 | **HeyGen** | 云端 API | 多模型网关 |
+| **ModelRunner** | 云端 API | 单密钥网关：Wan 2.7、Happy Horse（对白唇形同步）、Seedance Mini 低价档 |
 | **WAN 2.1** | 本地 GPU | 免费，提供 1.3B 和 14B 版本 |
 | **Hunyuan (混元)** | 本地 GPU | 免费，高质量 |
 | **CogVideo** | 本地 GPU | 免费，提供 2B 和 5B 版本 |
@@ -445,7 +447,7 @@ OpenMontage/
 </details>
 
 <details>
-<summary><strong>图像生成 — 10 种工具/提供商</strong></summary>
+<summary><strong>图像生成 — 11 种工具/提供商</strong></summary>
 
 | 提供商 | 类型 | 备注 |
 |----------|------|-------|
@@ -454,6 +456,7 @@ OpenMontage/
 | **Grok Imagine Image** | 云端 API | 强大的图像编辑、风格转换和多图合成 |
 | **GPT Image 2** | 云端 API | OpenAI 的图像模型 |
 | **Recraft** | 云端 API | 专注于设计的生成 |
+| **ModelRunner** | 云端 API | 单密钥接入 Seedream 5.0 Lite/Pro、Recraft V4.1、Stable Diffusion 3.5 |
 | **Local Diffusion** | 本地 GPU | Stable Diffusion，免费 |
 | **Pexels** | 素材库 | 免费的库存图片 |
 | **Pixabay** | 素材库 | 免费的库存图片 |
@@ -463,14 +466,15 @@ OpenMontage/
 </details>
 
 <details>
-<summary><strong>文本转语音 (TTS) — 4 家提供商</strong></summary>
+<summary><strong>文本转语音 (TTS) — 5 家提供商</strong></summary>
 
 | 提供商 | 类型 | 备注 |
 |----------|------|-------|
 | **ElevenLabs** | 云端 API | 顶级的语音质量 |
 | **Google TTS** | 云端 API | 700+ 种声音，50+ 种语言 — 最适合本地化 |
 | **OpenAI TTS** | 云端 API | 快速且价格实惠 |
 | **Piper** | 本地 | 完全免费，支持离线 |
+| **ModelRunner TTS** | 云端 API | 单密钥接入 Kokoro（亚美分级）、Gemini TTS（可导演式朗读）、ElevenLabs、Chatterbox |
 
 </details>
 
@@ -484,6 +488,7 @@ OpenMontage/
 | **Suno AI** | 云端 API | 生成带人声、歌词的完整歌曲，涵盖所有流派。长达 8 分钟。 |
 | **ElevenLabs Music** | 云端 API | AI 音乐生成 |
 | **ElevenLabs SFX** | 云端 API | 音效生成 |
+| **ModelRunner Music** | 云端 API | 单密钥接入 Lyria 2/3 音乐片段与 Stable Audio 2.5 长曲目 |
 
 **后期制作（始终可用，完全免费）：**