[Klaud Cold] minimaxm3-fp8-mi325x-vllm-agentic-mtp: MiniMax-M3 MI325X AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 MI325X AgentX EAGLE3 配方（黄金 AL 合成接受） - #2426
Conversation
…den-AL synthetic acceptance Add minimaxm3-fp8-mi325x-vllm-agentic-mtp, the spec-decoding=mtp variant of the MiniMax-M3 MXFP8 MI325X agentic-coding recipe: Inferact/MiniMax-M3-EAGLE3 draft head, 3 speculative tokens, no drafter backend pin (server-wide TRITON_ATTN), and synthetic acceptance pinned to the committed golden AL 2.83 (minimaxm3_eagle3.yaml, thinking_on, K=3). EVAL_ONLY keeps real verification. The pinned ROCm nightly postdates vllm-project/vllm#45546, so no in-place SupportsEagle3 patch is needed. 中文：新增 minimaxm3-fp8-mi325x-vllm-agentic-mtp，即 MiniMax-M3 MXFP8 MI325X 智能体 编码配方的投机解码（spec-decoding=mtp）变体：Inferact/MiniMax-M3-EAGLE3 草稿头、 3 个投机 token；服务端整体使用 TRITON_ATTN，drafter 无需固定注意力后端；合成接受 长度固定为黄金 AL 2.83（minimaxm3_eagle3.yaml，thinking_on，K=3）。EVAL_ONLY 保留 真实验证。所选 ROCm nightly 镜像晚于 vllm-project/vllm#45546，无需运行时补丁。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
中文：补充 perf-changelog 中 #2426 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
3 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
LGTM — mechanical addition of a new AgentX EAGLE3/MTP recipe variant following the repo's established pattern for sibling MI300X/MI355X MiniMax-M3 MTP recipes.
Extended reasoning...
Overview
Adds minimaxm3-fp8-mi325x-vllm-agentic-mtp: one new benchmark shell script (minimaxm3_fp8_mi325x_mtp.sh), one master-config entry in configs/amd-master.yaml, and one perf-changelog.yaml entry. No launcher or shared library changes. The new script is a near-verbatim copy of the already-merged non-MTP sibling (minimaxm3_fp8_mi325x.sh), with the only functional delta being the --speculative-config block (EAGLE3 draft head, 3 speculative tokens, synthetic-AL pinning for throughput runs, real verification for EVAL_ONLY).
Security risks
None. This is CI/benchmark infrastructure — a shell script that downloads public HF models, builds/caches Mooncake from a pinned upstream tag, and launches vLLM with fixed flags. No user-facing surface, no secrets, no auth/crypto changes.
Level of scrutiny
Low-to-moderate is appropriate. This is a config/benchmark-recipe addition, not production-critical code, and it follows an extremely well-established pattern already present dozens of times in amd-master.yaml (including near-identical MI300X/MI355X MiniMax-M3 EAGLE3/MTP variants). I spot-checked the specific factual claims in the PR description against the repo: golden_al_distribution/minimaxm3_eagle3.yaml does contain thinking_on: {3: 2.83}, matching the pinned synthetic_acceptance_length, and the new script's non-speculative serve flags are identical to the existing minimaxm3_fp8_mi325x.sh sibling.
Other factors
No bugs were reported by the automated bug-hunting pass, and I found no additional correctness issues in my own read of the diff. The search-space trimming and job-count math in the description are internally consistent with the YAML search-space entries. This is a config-only addition with no shared-code blast radius, so I'm comfortable approving without further human review.
|
Draft-head note for reviewers. This recipe uses Worth knowing: #2337 (merged 2026-07-29) moved B200/B300 fp8 to the 中文：本配方使用 需要说明的是：#2337（2026-07-29 合并）已将 B200/B300 fp8 切换到 |
…c grid steps >=2, stops at 16 Two corrections: - gpu-memory-utilization 0.90 (was 0.95, and 0.98 at TP4). The H100 twin died mid-warmup with torch.OutOfMemoryError on all 8 ranks (run 30515793863) at the non-MTP sibling's value; fixed_seq_len/minimaxm3_fp8_mi325x_mtp.sh passes no gmu flag at all, i.e. vLLM's 0.90 default. - Concurrency grid: GPU-resident [1,4,8,12,16], Mooncake [4,8,12,16], DEP [8,12,16]. Every step is >=2 and every arm stops hard at 16; 74 jobs -> 34. 中文：两处修正：(1) gpu-memory-utilization 改为 0.90（原为 0.95，TP4 为 0.98）—— 本配方的 H100 版本在非 MTP 取值下于预热阶段 8 个 rank 全部 OOM（run 30515793863）， 而 fixed_seq_len/minimaxm3_fp8_mi325x_mtp.sh 未设置该参数（即 vLLM 默认 0.90）。 (2) 并发网格：GPU 常驻 [1,4,8,12,16]、Mooncake [4,8,12,16]、DEP [8,12,16]；相邻 并发间隔至少为 2，且一律止于 16，作业数由 74 降至 34。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30515954247 |
…on AMD
Mooncake is not used on AMD. The HIP-built transfer engine runs over TCP on
these clusters and the store collapses mid-replay ("ServerSession::readBody
failed ... Connection reset by peer" until AIPerf aborts), as the MI300X sibling
hit in run 30517360090.
Remove all three host-DRAM arms and the in-script Mooncake build/launch
plumbing. The recipe is GPU-resident only (TP4, TP4/EP4, TP8, TP8/EP8, TP8/DEP8;
34 jobs -> 23) and hard-fails if a dram row is ever pointed at it.
中文：AMD 上不再使用 Mooncake —— 基于 HIP 构建的传输引擎在这些集群上走 TCP，运行中
store 会崩溃（持续输出 "ServerSession::readBody failed ... Connection reset by peer"
直至 AIPerf 中止），MI300X 版本已在 run 30517360090 中复现。删除全部三条主机 DRAM
分支及脚本内的 Mooncake 构建/启动逻辑，改为仅 GPU 常驻（TP4、TP4/EP4、TP8、TP8/EP8、
TP8/DEP8，作业数由 34 降至 23）；若仍传入 dram 配置将直接报错退出。
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…325x-agentic-mtp # Conflicts: # perf-changelog.yaml
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517335728 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30518679416 |
1 participant
Summary
Adds
minimaxm3-fp8-mi325x-vllm-agentic-mtp— the EAGLE3 speculative-decoding (spec-decoding: mtp) variant of the MiniMax-M3 MXFP8 MI325X (gfx942) AgentX recipe. One new benchmark script, one master-config entry, one changelog entry;launch_mi325x-amds.shalready composesagentic/+_mtp, so no launcher change.Speculative config —
method: eagle3onInferact/MiniMax-M3-EAGLE3,num_speculative_tokens: 3. No drafterattention_backendoverride, unlike the CUDA recipes: the FlashInfer "page size 128 requires GQA/MQA" limitation that forcesFLASH_ATTNfor the MHA EAGLE3 head is FlashInfer-specific, and this server runsTRITON_ATTNthroughout — the same reasoningfixed_seq_len/minimaxm3_fp8_mi325x_mtp.shdocuments.Synthetic forced AL — throughput runs set
rejection_sample_method: syntheticwithsynthetic_acceptance_length: 2.83, the committed golden AL for MiniMax-M3 EAGLE3 at K=3,thinking_on(golden_al_distribution/minimaxm3_eagle3.yaml), per the AgentX policy ingolden_al_distribution/README.md. Theminimaxm3_eagle3_gqa.yamlcurve belongs to theInferact/MiniMax-M3-EAGLE3-GQAdraft (run 29784780049) and is not mixed in.EVAL_ONLYruns drop synthetic acceptance and keep real target verification.No in-place SupportsEagle3 patch.
vllm/vllm-openai-rocm:nightly-04c2a8deac44fdb1ca3e2b5ec3e6bf16f3f6a914is built from upstream 2026-06-23, after vllm-project/vllm#45546 (EAGLE3 on the AMD MiniMax-M3 model) merged on 2026-06-14. Only the oldervllm-openai-rocm:minimax-m3bring-up image still needs the runtime patch that the fixed-seq-len MI325X recipe carries.Serve shape is otherwise identical to the non-MTP agentic sibling (Mooncake host-DRAM KV offload over TCP with the HIP-built transfer engine,
--block-size 128,--language-model-only,--kv-cache-dtype fp8,--attention-backend TRITON_ATTN, gmu 0.90 (the 0.98 TP4 bump dropped),minimax_m3parsers,VLLM_USE_BREAKABLE_CUDAGRAPH=0,max-num-seqs 2*CONC). Cudagraph capture stays at the sibling default rather than being raised toMAX_NUM_SEQS*(1+3)tokens as on the CUDA agentic MTP recipes, matching the already-merged ROCm MiniMax-M3 MTP recipes.Search space is TP4, TP4/EP4, TP8, TP8/EP8 and TP8/DEP8, GPU-resident only on the AgentX MTP concurrency grid:
[1, 4, 8, 12, 16](DEP/dp-attn[8, 12, 16]). Every step is at least 2 concurrency apart and every arm stops hard at conc 16. The non-MTP entry's three Mooncake host-DRAM arms are dropped — Mooncake is not used on AMD (see the revision note below). 23 generated jobs.Validation:
bash -nclean; both YAMLs parse;generate_sweep_configs.py test-configemits 23 agentic-coding configs withspec-decoding: mtponcluster:mi325x-amds.中文说明
新增
minimaxm3-fp8-mi325x-vllm-agentic-mtp：MiniMax-M3 MXFP8 MI325X（gfx942）AgentX（智能体编码）配方的 EAGLE3 投机解码（spec-decoding: mtp）变体。包含一个新脚本、一条 master 配置条目与一条 changelog 条目；launch_mi325x-amds.sh已能拼出agentic/+_mtp路径，启动器无需改动。投机解码配置：
method: eagle3，草稿头Inferact/MiniMax-M3-EAGLE3，num_speculative_tokens: 3。与 CUDA 配方不同，drafter 无需固定注意力后端：迫使 CUDA 侧为 MHA EAGLE3 头指定FLASH_ATTN的「page size 128 需要 GQA/MQA」限制是 FlashInfer 特有的，而这里服务端整体运行 TRITON_ATTN（与fixed_seq_len/minimaxm3_fp8_mi325x_mtp.sh的说明一致）。合成强制 AL：吞吐量测试设置
rejection_sample_method: synthetic与synthetic_acceptance_length: 2.83，即 MiniMax-M3 EAGLE3 在 K=3、thinking_on 下已提交的黄金 AL（golden_al_distribution/minimaxm3_eagle3.yaml），遵循golden_al_distribution/README.md的 AgentX 规则。minimaxm3_eagle3_gqa.yaml属于Inferact/MiniMax-M3-EAGLE3-GQA草稿头（run 29784780049），两条曲线不混用。EVAL_ONLY关闭合成接受、保留真实验证。无需运行时 SupportsEagle3 补丁：
vllm/vllm-openai-rocm:nightly-04c2a8deac44fdb1ca3e2b5ec3e6bf16f3f6a914基于上游 2026-06-23 的提交，晚于 2026-06-14 合入的 vllm-project/vllm#45546。只有较旧的vllm-openai-rocm:minimax-m3镜像仍需该补丁。服务参数与非 MTP 版本一致（基于 HIP 构建的 Mooncake TCP 主机 DRAM KV 卸载、
--block-size 128、--language-model-only、--kv-cache-dtype fp8、--attention-backend TRITON_ATTN、gmu 0.90（不再对 TP4 提升至 0.98）、minimax_m3解析器、VLLM_USE_BREAKABLE_CUDAGRAPH=0、max-num-seqs 2*CONC）。cudagraph 捕获上限保持与非 MTP 版本相同，未像 CUDA 侧那样提高到MAX_NUM_SEQS*(1+3)token，这与已合并的 ROCm MiniMax-M3 MTP 配方一致。搜索空间为仅 GPU 常驻的 TP4、TP4/EP4、TP8、TP8/EP8 与 TP8/DEP8，采用 AgentX MTP 并发网格
[1, 4, 8, 12, 16]（DEP/dp-attn 为[8, 12, 16]）。相邻并发间隔至少为 2，且一律止于并发 16。非 MTP 条目中的 Mooncake 主机 DRAM 分支已全部删除 —— AMD 上不使用 Mooncake（见下方修订说明）。共生成 23 个作业。验证：
bash -n通过；两个 YAML 均可解析；generate_sweep_configs.py test-config生成 23 条cluster:mi325x-amds上的 agentic-coding 配置。🤖 Generated with Claude Code
Revision (post-first-sweep)
--gpu-memory-utilization0.90. The first agentic job on [Klaud Cold] minimaxm3-fp8-h100-vllm-agentic-mtp: MiniMax-M3 H100 AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 H100 AgentX EAGLE3 配方（黄金 AL 合成接受） #2423 (TP8/EP8 c9, run 30515793863) died mid-warmup: all 8 ranks raisedtorch.OutOfMemoryError96 MiB short, killing the engine core (EngineDeadError) and tripping AIPerf's terminal warmup failure. The recipe had inherited the non-MTP agentic sibling's gmu, which has no draft head to house; the proven MTP sibling on each of these SKUs runs 0.90 (explicit on H100/H200, vLLM's default on the ROCm ones). Applied to all four MiniMax PRs.中文：(1)
--gpu-memory-utilization改为 0.90 —— #2423 的首个智能体作业（TP8/EP8 c9，run 30515793863）在预热阶段 8 个 rank 全部 OOM（仅差 96 MiB），导致 engine core 崩溃并触发 AIPerf 终止性预热失败；此前沿用了非 MTP 版本的取值，而该版本无需为草稿头预留显存，各 SKU 上已验证的 MTP 配置均为 0.90。四个 MiniMax PR 均已修正。(2) 并发网格：相邻间隔至少为 2，且一律止于 16。Revision 2 — no Mooncake on AMD
The MI300X dram-offload job (
TP8 mtp mooncake c8, run 30517360090) failed. The HIP-built Mooncake transfer engine runs over TCP on these clusters and the store collapses mid-replay, repeatingServerSession::readBody failed ... Connection reset by peeruntil AIPerf aborts. Per maintainer direction, Mooncake is not used on AMD: all host-DRAM arms are removed from this recipe along with the in-script Mooncake build/launch plumbing, and the script now hard-fails if adramrow is ever pointed at it rather than silently serving GPU-resident under a dram label. The Qwen3.5 AMD AgentX PRs are unaffected — they use SGLang HiCache, not Mooncake.中文：MI300X 的 dram 卸载作业（
TP8 mtp mooncake c8，run 30517360090）失败 —— 基于 HIP 构建的 Mooncake 传输引擎在这些集群上走 TCP，运行中 store 会崩溃，持续输出ServerSession::readBody failed ... Connection reset by peer直至 AIPerf 中止。按维护者要求，AMD 上不使用 Mooncake：本配方删除全部主机 DRAM 分支及脚本内的 Mooncake 构建/启动逻辑；若仍传入dram配置将直接报错退出，而不会在 dram 标签下静默执行 GPU 常驻测试。Qwen3.5 的 AMD AgentX PR 不受影响 —— 它们使用的是 SGLang HiCache，而非 Mooncake。