[Klaud Cold] minimaxm3-fp8-h100-vllm-agentic-mtp: MiniMax-M3 H100 AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 H100 AgentX EAGLE3 配方（黄金 AL 合成接受） - #2423
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
中文：补充 perf-changelog 中 #2423 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Draft-head note for reviewers. This recipe uses Worth knowing: #2337 (merged 2026-07-29) moved B200/B300 fp8 to the 中文：本配方使用 需要说明的是：#2337（2026-07-29 合并）已将 B200/B300 fp8 切换到 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| --tool-call-parser minimax_m3 \ | ||
| --reasoning-parser minimax_m3 \ |
There was a problem hiding this comment.
🔴 Lines 175-176 pass both --tool-call-parser minimax_m3 and --reasoning-parser minimax_m3, but minimaxm3_fp4_mi355x.sh (merged the day before, PR #2118) intentionally omits the reasoning parser for this exact agentic-coding scenario, with a detailed comment explaining that MiniMax-M3's <mm:think> block must round-trip in message.content every turn or multi-turn quality collapses into repeated commands / empty patches. This is inherited from minimaxm3_fp8_h100.sh and its h200/mi300x/mi325x siblings, but this PR copies the same known-bad flag pair into brand-new code, and it hits hardest on the EVAL_ONLY real-verification path (lines 195-196), which would score near zero from the resulting degeneration. Fix: drop --reasoning-parser minimax_m3 here as was already done in the mi355x recipe.
Extended reasoning...
The bug: minimaxm3_fp8_h100_mtp.sh:175-176 passes both --tool-call-parser minimax_m3 and --reasoning-parser minimax_m3 to the vLLM server. MiniMax-M3 is an interleaved-thinking model that emits an <mm:think>...</mm:think> block as part of its response. When --reasoning-parser minimax_m3 is enabled, vLLM strips that block out of message.content and surfaces it instead as a separate, response-only reasoning_content field on the OpenAI-compatible completion object.
Why that breaks multi-turn agentic runs: the agentic-coding harness this script drives (run_agentic_replay_and_write_outputs, mini-swe-agent + litellm's OpenAI client) only round-trips message.content back into the conversation on the next turn — it does not resend reasoning_content. So with the reasoning parser enabled, the model's think block silently disappears from its own conversation history every turn. Without visibility into its prior plan, the model degenerates: it repeats the same command turn after turn until the step limit is hit, producing an empty patch.
Why nothing here prevents it: this exact failure mode and fix are already documented in the repo. benchmarks/single_node/agentic/minimaxm3_fp4_mi355x.sh:119-127 (PR #2118, merged one day before this recipe's non-MTP sibling) keeps --tool-call-parser minimax_m3 but explicitly omits --reasoning-parser minimax_m3, with an inline comment spelling out precisely this mechanism. That fix was never backported to minimaxm3_fp8_h100.sh (nor its h200/mi300x/mi325x siblings), and this new MTP recipe copies the non-MTP h100 sibling's server flags verbatim (including both parsers), propagating the known-bad configuration into new code rather than following the already-established fix.
Impact: the failure is model/harness-level, not hardware- or spec-decode-specific, so it applies identically on this H100 EAGLE3 recipe. The throughput sweep runs use rejection_sample_method: synthetic with a forced golden acceptance length, so the actual generated text there is synthetic/garbage by design and the parser bug doesn't change the throughput numbers. But the EVAL_ONLY path (lines 195-196) drops synthetic acceptance specifically to do real target verification and score genuine agentic quality — this is exactly the path that would collapse into empty-patch degeneration from the stripped think block, defeating the purpose of carving out EVAL_ONLY in the first place.
Step-by-step proof of the mechanism:
- Server starts with
--reasoning-parser minimax_m3(line 176), so every completion response has its<mm:think>...</mm:think>content removed frommessage.contentand placed only inmessage.reasoning_content. - Turn N: the agent (mini-swe-agent via litellm's OpenAI-compatible client) receives a response, appends it to its running message list. It appends
message.content(the tool-call/action portion) —reasoning_contentis not part of the OpenAI chat schema the client resends, so it's dropped. - Turn N+1: the server serves the next completion using only the truncated history — the model has no record of its own prior
<mm:think>reasoning/plan. - Because MiniMax-M3 relies on that block persisting in-context to track multi-step plans (per the mi355x comment), the model re-derives a similar-looking next step, often repeating its previous command verbatim.
- This repeats every turn until the step limit, producing no net progress and an empty final patch — which the
EVAL_ONLYreal-verification path would score at or near zero, even though the server and harness are otherwise functioning correctly.
Fix: drop --reasoning-parser minimax_m3 from the vllm serve invocation (keep --tool-call-parser minimax_m3), matching what minimaxm3_fp4_mi355x.sh already does for this identical scenario.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30515793863 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517270909 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517356982 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517356982 |
中文：补充 perf-changelog 中 #2423 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
c8b3cef to
0034507
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31142073773 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31142073773 |
…n-AL synthetic acceptance Add minimaxm3-fp8-h100-vllm-agentic-mtp, the spec-decoding=mtp variant of the MiniMax-M3 MXFP8 H100 agentic-coding recipe: Inferact/MiniMax-M3-EAGLE3 draft head, 3 speculative tokens, FLASH_ATTN drafter, and synthetic acceptance pinned to the committed golden AL 2.83 (minimaxm3_eagle3.yaml, thinking_on, K=3). EVAL_ONLY keeps real target verification. Serve shape, Mooncake offload and search-space arms mirror the non-MTP sibling, trimmed at the top concurrencies. 中文：新增 minimaxm3-fp8-h100-vllm-agentic-mtp，即 MiniMax-M3 MXFP8 H100 智能体 编码配方的投机解码（spec-decoding=mtp）变体：使用 Inferact/MiniMax-M3-EAGLE3 草稿头、3 个投机 token、drafter 固定为 FLASH_ATTN，并将合成接受长度固定为已提交 的黄金 AL 2.83（minimaxm3_eagle3.yaml，thinking_on，K=3）。EVAL_ONLY 场景保留真实 目标验证。服务参数、Mooncake KV 卸载与搜索空间与非 MTP 版本保持一致，仅在高并发端 做了裁剪。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
中文：补充 perf-changelog 中 #2423 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to fit the EAGLE3 draft The first agentic job (TP8/EP8 c9, run 30515793863) died mid-warmup: every one of the 8 ranks raised torch.OutOfMemoryError 96 MiB short, killing the engine core (EngineDeadError) and tripping AIPerf's terminal warmup failure. The recipe inherited --gpu-memory-utilization 0.95 from the non-MTP agentic sibling, which has no draft head to house. fixed_seq_len/minimaxm3_fp8_h100_mtp.sh — the proven MTP configuration on this SKU — runs 0.90. Match it. 中文：首个智能体作业（TP8/EP8 c9，run 30515793863）在预热阶段失败：8 个 rank 全部 抛出 torch.OutOfMemoryError（仅差 96 MiB），导致 engine core 崩溃（EngineDeadError） 并触发 AIPerf 的终止性预热失败。该配方沿用了非 MTP 版本的 --gpu-memory-utilization 0.95，而后者并不需要为草稿头预留显存。本 SKU 上已验证的 MTP 配置 fixed_seq_len/minimaxm3_fp8_h100_mtp.sh 使用 0.90，现与之对齐。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the dense integer sampling inherited from the non-MTP agentic siblings with the AgentX MTP grid: GPU-resident [1,4,8,12,16], offload [4,8,12,16]. Single-step sampling cannot separate configs by more than noise on the agentic corpus, and past conc 16 these SKUs are in the post-cliff thrashing regime. 中文：将沿用自非 MTP 智能体配方的逐 1 递增并发采样，替换为 AgentX MTP 并发网格： GPU 常驻 [1,4,8,12,16]，KV 卸载 [4,8,12,16]。相邻并发间隔至少为 2，且一律止于 16 —— 逐 1 采样在智能体语料上无法区分出超过噪声的差异，并发超过 16 后这些 SKU 已进入 显存瓶颈后的抖动区间。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
0034507 to
875fe9d
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31204980135 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31204980135 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31204980135 |
# Conflicts: # perf-changelog.yaml
hpc-gpu-1-1 kills jobs seconds after they land on it. Job 14594 (h100-dgxc-slurm_18) is the example: allocated 01:48:50, step .1 FAILED at 01:49:06, whole allocation cancelled 16s in. Any sweep cell scheduled there burns a matrix slot without producing a result. Adds --exclude to salloc, gated on H100_SLURM_EXCLUDED_NODELIST so the node can be brought back by exporting the variable empty -- the same shape launch_b300-nv.sh already uses for b300-018. The exclusion goes on salloc rather than srun because srun runs inside the allocation, so the node set is fixed by then. Built as an array so an empty override expands to nothing instead of a bare --exclude=. 中文：在 H100 DGXC 的 Slurm 分配中排除 hpc-gpu-1-1。该节点会在作业落上去后数秒内使其 失败——作业 14594（h100-dgxc-slurm_18）于 01:48:50 分配，步骤 .1 在 01:49:06 失败， 整个分配在 16 秒内被取消；任何被调度到该节点的扫描单元都会白白占用一个矩阵槽位。 在 salloc 上添加 --exclude，并通过 H100_SLURM_EXCLUDED_NODELIST 控制（将该变量导出为 空即可恢复使用该节点），与 launch_b300-nv.sh 处理 b300-018 的方式一致。排除项置于 salloc 而非 srun，因为 srun 在该分配内运行，此时节点集合已确定；参数以数组形式构造， 空值时不会残留裸的 --exclude=。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31237535535 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31237535535 |
1 participant
Summary
Adds
minimaxm3-fp8-h100-vllm-agentic-mtp— the EAGLE3 speculative-decoding (spec-decoding: mtp) variant of the MiniMax-M3 MXFP8 H100 AgentX recipe. One new benchmark script, one master-config entry, one changelog entry; no launcher change is needed (launch_h100-dgxc-slurm.shalready composesagentic/+_mtp).Speculative config —
method: eagle3onInferact/MiniMax-M3-EAGLE3,num_speculative_tokens: 3, drafter pinned toFLASH_ATTN(the EAGLE3 head is MHA; FlashInfer only serves the mandatory page size 128 via its trtllm-gen kernel, which needs GQA/MQA). Same draft and level as every merged MiniMax-M3 MTP recipe.Synthetic forced AL — throughput runs set
rejection_sample_method: syntheticwithsynthetic_acceptance_length: 2.83, the committed golden AL for MiniMax-M3 EAGLE3 at K=3,thinking_on(golden_al_distribution/minimaxm3_eagle3.yaml). Per the AgentX policy ingolden_al_distribution/README.md, a submission may choose any supported draft length but may not substitute a different acceptance target. The separateminimaxm3_eagle3_gqa.yamlcurve belongs to theInferact/MiniMax-M3-EAGLE3-GQAdraft (run 29784780049) and is not mixed in.EVAL_ONLYruns drop synthetic acceptance and keep real verification, since synthetic acceptance commits drafted tokens regardless of the target logits and would score the eval at ~0.Serve shape is otherwise identical to the non-MTP agentic sibling (Mooncake host-DRAM KV offload,
--block-size 128,--language-model-only,--kv-cache-dtype fp8,--attention-backend TRITON_ATTN,minimax_m3parsers, gmu 0.90,--cpu-offload-gb 26, vllm-router for DP-attention) so the spec-decode delta is readable at equal concurrency.max-num-seqsstays2*CONCfor AgentX subagent fan-out; cudagraph capture is raised toMAX_NUM_SEQS*(1+3)tokens because vLLM's FULL_DECODE_ONLY ladder is token-denominated.Search space covers four arms (TP8 and TP8/EP8 × GPU-resident and Mooncake) on the AgentX MTP concurrency grid: GPU-resident rows
[1, 4, 8, 12, 16], KV-offload rows[4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 18 generated jobs.Validation:
bash -nclean; both YAMLs parse;generate_sweep_configs.py test-configemits 18 agentic-coding configs withspec-decoding: mtpand the expected TP/EP/KV arms.中文说明
新增
minimaxm3-fp8-h100-vllm-agentic-mtp：MiniMax-M3 MXFP8 H100 AgentX（智能体编码）配方的 EAGLE3 投机解码（spec-decoding: mtp）变体。包含一个新的基准测试脚本、一条 master 配置条目和一条 changelog 条目；启动器无需改动（launch_h100-dgxc-slurm.sh已能拼出agentic/+_mtp路径）。投机解码配置：
method: eagle3，草稿头Inferact/MiniMax-M3-EAGLE3，num_speculative_tokens: 3，drafter 固定FLASH_ATTN（EAGLE3 头是 MHA，而 FlashInfer 仅通过要求 GQA/MQA 的 trtllm-gen kernel 支持强制的 page size 128）。与仓库中所有已合并的 MiniMax-M3 MTP 配方使用相同的草稿头与投机层数。合成强制 AL：吞吐量测试设置
rejection_sample_method: synthetic与synthetic_acceptance_length: 2.83，即 MiniMax-M3 EAGLE3 在 K=3、thinking_on 下已提交的黄金 AL（golden_al_distribution/minimaxm3_eagle3.yaml）。依据golden_al_distribution/README.md的 AgentX 规则，提交方可自选草稿长度，但不得替换接受长度目标。minimaxm3_eagle3_gqa.yaml对应的是Inferact/MiniMax-M3-EAGLE3-GQA草稿头（run 29784780049），两条曲线不混用。EVAL_ONLY场景关闭合成接受、保留真实验证，否则生成文本错误、评估分数接近 0。服务参数与非 MTP 版本完全一致（Mooncake 主机 DRAM KV 卸载、
--block-size 128、--language-model-only、--kv-cache-dtype fp8、--attention-backend TRITON_ATTN、minimax_m3解析器、gmu 0.90、--cpu-offload-gb 26、DP-attention 下的 vllm-router），便于在相同并发下直接对比投机解码带来的差异。max-num-seqs保持2*CONC以容纳 AgentX 子智能体扇出；cudagraph 捕获上限提高到MAX_NUM_SEQS*(1+3)个 token，因为 vLLM 的 FULL_DECODE_ONLY 阶梯是按 token 计的。搜索空间覆盖四条分支（TP8 与 TP8/EP8 × GPU 常驻与 Mooncake），并采用 AgentX MTP 并发网格：GPU 常驻
[1, 4, 8, 12, 16]，KV 卸载[4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 18 个作业。验证：
bash -n通过；两个 YAML 均可解析；generate_sweep_configs.py test-config生成 18 条 agentic-coding 配置，spec-decoding: mtp与 TP/EP/KV 分支均符合预期。🤖 Generated with Claude Code
Revision (post-first-sweep)
--gpu-memory-utilization0.90. The first agentic job on [Klaud Cold] minimaxm3-fp8-h100-vllm-agentic-mtp: MiniMax-M3 H100 AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 H100 AgentX EAGLE3 配方（黄金 AL 合成接受） #2423 (TP8/EP8 c9, run 30515793863) died mid-warmup: all 8 ranks raisedtorch.OutOfMemoryError96 MiB short, killing the engine core (EngineDeadError) and tripping AIPerf's terminal warmup failure. The recipe had inherited the non-MTP agentic sibling's gmu, which has no draft head to house; the proven MTP sibling on each of these SKUs runs 0.90 (explicit on H100/H200, vLLM's default on the ROCm ones). Applied to all four MiniMax PRs.中文：(1)
--gpu-memory-utilization改为 0.90 —— #2423 的首个智能体作业（TP8/EP8 c9，run 30515793863）在预热阶段 8 个 rank 全部 OOM（仅差 96 MiB），导致 engine core 崩溃并触发 AIPerf 终止性预热失败；此前沿用了非 MTP 版本的取值，而该版本无需为草稿头预留显存，各 SKU 上已验证的 MTP 配置均为 0.90。四个 MiniMax PR 均已修正。(2) 并发网格：相邻间隔至少为 2，且一律止于 16。