Add inclusionAI/Ling-3.0-flash-FP8 recipe

[zexplorerhj]

Summary

Adds a serving recipe for inclusionAI/Ling-3.0-flash-fp8.

• Architecture: BailingMoeV3ForCausalLM, a hybrid MLA/KDA MoE model.
• Size: 124B total / 5.5B active parameters, with 512 routed experts
  (8 active per token), one shared expert, and a 3.1B MTP layer.
• Context: 256K tokens.
• Precision: serialized block-FP8 weights with BF16 compute.
• Features: CUDA graphs, prefix caching, Triton MoE, Ling3
  reasoning/tool-call parsing, and opt-in MTP speculative decoding.
• Default deployment: single-node TP=2 on NVIDIA H200.
• Additional validated deployment: single-node TP=4 with expert parallelism
  on NVIDIA H200.

The recipe requires a vLLM build containing Ling-3.0-flash serialized
block-FP8 support. Expert parallelism is not enabled by default.

The MTP configuration is opt-in and uses num_speculative_tokens=3.

Test Plan

• node scripts/build-recipes-api.mjs
• Verify the generated recommended command defaults to TP=2, keeps the
  Triton MoE backend, and does not enable expert parallelism.
• Start the TP=2 FP8 recipe on NVIDIA H200.
• Start the TP=4+EP FP8 configuration on NVIDIA H200.
• Verify base generation, ling3 reasoning parsing, automatic tool calls,
  and opt-in MTP decoding.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
vllm-recipes	Ready	Preview	Aug 6, 2026 12:53pm

[gemini-code-assist]

Code Review

This pull request introduces a new model configuration file for inclusionAI/Ling-3.0-flash-FP8, detailing its metadata, deployment arguments, and usage guide. The review feedback highlights a few inconsistencies: the launch command and Python example use an uppercase model ID (inclusionAI/Ling-3.0-flash-FP8) instead of the correct lowercase ID (inclusionAI/Ling-3.0-flash-fp8), which can cause download issues due to case sensitivity. Additionally, the context_length in the model metadata is set to 262144, contradicting the 131072 limit documented in the guide and base recipe.

[xianbaoqian]

Instead of having two files for each quantization, you can can just add a variant to avoid duplication :)

Example:

recipes/models/moonshotai/Kimi-K3.yaml

Line 97 in e70edaa

variants: