[Klaud Cold] minimaxm3-fp8-h200-vllm-agentic-mtp: MiniMax-M3 H200 AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 H200 AgentX EAGLE3 配方（黄金 AL 合成接受） - #2424
Conversation
中文：补充 perf-changelog 中 #2424 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Draft-head note for reviewers. This recipe uses Worth knowing: #2337 (merged 2026-07-29) moved B200/B300 fp8 to the 中文：本配方使用 需要说明的是：#2337（2026-07-29 合并）已将 B200/B300 fp8 切换到 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline pr-link finding, I also checked whether the GPU-resident search-space arms (conc-list ending ...11, 12, 14, 16) silently skip conc=13 as a bug — it's consistent with how these lists are manually curated per arm (the non-MTP sibling's list also omits conc=3), not a generation defect, so I'm not flagging it.
Extended reasoning...
Bugs were already found and reported inline (perf-changelog.yaml pr-link left as TBD), which already signals this PR needs human attention before merge. I independently confirmed that finding against git show HEAD:perf-changelog.yaml, which does show pr-link: TBD as the last line, consistent with the inline report. Beyond that, I reviewed the candidate issue that finder agents raised and verifiers ruled out (GPU-resident search-space arms silently skipping conc=13 in configs/nvidia-master.yaml) and confirmed it is a benign, manually-curated concurrency list rather than a bug, matching the pattern of gaps already present in the non-MTP sibling entry. I'm posting this brief note only to record that this specific candidate was checked, per the exception process for bugs-found reviews.
| - "Speculative config: method eagle3 on the Inferact/MiniMax-M3-EAGLE3 draft head, num_speculative_tokens 3, drafter pinned to attention_backend FLASH_ATTN -- the same draft, level and pin as the merged MiniMax-M3 CUDA MTP recipes (the EAGLE3 head is MHA and FlashInfer only serves the mandatory page size 128 through its GQA/MQA-only trtllm-gen kernel)." | ||
| - "Throughput runs pin rejection_sample_method synthetic with synthetic_acceptance_length 2.83 -- the committed golden AL for MiniMax-M3 EAGLE3 at K=3, thinking_on (golden_al_distribution/minimaxm3_eagle3.yaml), per the AgentX policy in golden_al_distribution/README.md. The minimaxm3_eagle3_gqa.yaml curve was measured against the Inferact/MiniMax-M3-EAGLE3-GQA draft (run 29784780049) and is not mixed in. EVAL_ONLY runs drop synthetic acceptance and keep real target verification." | ||
| - "Serve shape is otherwise identical to the non-MTP agentic sibling (Mooncake host-DRAM KV offload, --block-size 128, --language-model-only, --kv-cache-dtype fp8, --attention-backend TRITON_ATTN, gpu-memory-utilization 0.92, minimax_m3 parsers, vllm-router for DP-attention), with max-num-seqs 2*CONC for AgentX subagent fan-out and cudagraph capture raised to MAX_NUM_SEQS*(1+3) TOKENS for vLLM's token-denominated FULL_DECODE_ONLY ladder." | ||
| - "Search space mirrors the non-MTP entry's three arms (TP8 and TP8/EP8 GPU-resident, TP8/EP8 Mooncake) trimmed at the extreme-conc end: GPU-resident 1-16 (vs 1-20) and Mooncake 5-18 (vs 5-20). The 269k-token service-time-weighted request already places the H200 GPU-resident cliff near conc 10, and the draft head plus draft KV tighten that budget further." |
There was a problem hiding this comment.
🔴 The final perf-changelog.yaml entry for this PR has pr-link: TBD instead of https://github.com/SemiAnalysisAI/InferenceX/pull/2424 — verified directly against HEAD's committed content, not just the diff. This isn't only a style deviation: utils/validate_perf_changelog.py's PR_LINK_PLACEHOLDERS only accepts XXX or the exact real URL, so the standard merge path (utils/merge_with_reuse.sh → prepare_perf_changelog_merge.py canonicalize) will raise ChangelogValidationError and abort under set -e until this is corrected to the real PR URL (or the XXX placeholder, which the tooling auto-backfills).
Extended reasoning...
What the bug is: The changelog entry appended by this PR ends with pr-link: TBD rather than a real PR URL or a recognized placeholder. I verified this directly with git show HEAD:perf-changelog.yaml | tail, which shows the literal committed content of perf-changelog.yaml at the tip of this PR's branch — the last line is pr-link: TBD. Every other one of the ~658 prior entries in this file uses pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/<N>, matching the convention documented in AGENTS.md.
Why the preloaded diff looked fine: The diff shown in this review's preloaded context happens to render the last line as pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2424. This is a discrepancy between the cached/preloaded diff text and the actual committed blob at HEAD. Since git show HEAD:perf-changelog.yaml reads the real git object for the commit under review (5ed3a3d), it is authoritative over a preloaded diff rendering — and it unambiguously contains TBD, not the pull/2424 URL. I re-ran this check independently before filing to rule out a caching artifact on my end, and the result was consistent.
Why nothing upstream of merge catches it: validation.py's ChangelogEntry.pr_link field is a bare str with no URL/format validation, so generate_sweep_configs.py and friends won't flag it at config-generation time. The CI PR-gate script (process_changelog.py) also doesn't check the value of pr-link on new entries. So a normal CI run against this branch would appear to pass.
Where it actually breaks: The repo's standard merge tooling does check it. utils/merge_with_reuse.sh unconditionally invokes prepare_perf_changelog_merge.py canonicalize under set -e. That script's canonicalize_appended_links → compare_entries → validate_added_pr_link path rejects any appended pr-link value that is neither in PR_LINK_PLACEHOLDERS ({\"XXX\", \"https://github.com/SemiAnalysisAI/InferenceX/pull/XXX\"}) nor the exact expected URL for the PR being merged. TBD matches neither, so validate_added_pr_link raises ChangelogValidationError, which aborts the merge script.
Step-by-step proof:
- Checkout/fetch this PR's branch at commit 5ed3a3d.
- Run
git show HEAD:perf-changelog.yaml | tail -1→ printspr-link: TBD. - Run the repo's normal merge path:
utils/merge_with_reuse.sh 2424(or whatever the real merge invocation is for this PR number). - That script calls
prepare_perf_changelog_merge.py canonicalize, which loads the new entry, seespr-link: TBD, and callsvalidate_added_pr_link(\"TBD\", 2424). TBDis not inPR_LINK_PLACEHOLDERSand is not equal tohttps://github.com/SemiAnalysisAI/InferenceX/pull/2424, so the function raisesChangelogValidationError.- The script is running under
set -e, so the merge aborts with a non-zero exit before completing.
The fix is a one-line change: replace pr-link: TBD with either the real URL https://github.com/SemiAnalysisAI/InferenceX/pull/2424, or the XXX placeholder that the canonicalizer will auto-backfill with the correct URL at merge time.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30515848324 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517286304 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517358870 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=30517358870 |
中文：补充 perf-changelog 中 #2424 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
088ef51 to
4d7e99f
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31142043539 |
2 similar comments
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31142043539 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31142043539 |
…n-AL synthetic acceptance Add minimaxm3-fp8-h200-vllm-agentic-mtp, the spec-decoding=mtp variant of the MiniMax-M3 MXFP8 H200 agentic-coding recipe: Inferact/MiniMax-M3-EAGLE3 draft head, 3 speculative tokens, FLASH_ATTN drafter, and synthetic acceptance pinned to the committed golden AL 2.83 (minimaxm3_eagle3.yaml, thinking_on, K=3). EVAL_ONLY keeps real target verification. Serve shape, Mooncake offload and search-space arms mirror the non-MTP sibling, trimmed at the top concurrencies. 中文：新增 minimaxm3-fp8-h200-vllm-agentic-mtp，即 MiniMax-M3 MXFP8 H200 智能体 编码配方的投机解码（spec-decoding=mtp）变体：Inferact/MiniMax-M3-EAGLE3 草稿头、 3 个投机 token、drafter 固定为 FLASH_ATTN，合成接受长度固定为黄金 AL 2.83 （minimaxm3_eagle3.yaml，thinking_on，K=3）。EVAL_ONLY 保留真实目标验证。服务参数、 Mooncake KV 卸载与搜索空间沿用非 MTP 版本，仅在高并发端裁剪。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
中文：补充 perf-changelog 中 #2424 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…to fit the EAGLE3 draft The H100 twin of this recipe died mid-warmup with torch.OutOfMemoryError on all 8 ranks (run 30515793863) because it inherited the non-MTP agentic sibling's gpu-memory-utilization, which leaves no headroom for the draft head and its KV. Apply the same correction here pre-emptively: 0.90, matching fixed_seq_len/minimaxm3_fp8_h200_mtp.sh, the proven MTP setting on this SKU. 中文：本配方的 H100 版本在预热阶段因 8 个 rank 全部 OOM 而失败（run 30515793863）， 原因是沿用了非 MTP 版本的 gpu-memory-utilization，未给草稿头及其 KV 预留显存。此处 预先做同样的修正：改为 0.90，与本 SKU 上已验证的 MTP 配置 fixed_seq_len/minimaxm3_fp8_h200_mtp.sh 保持一致。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the dense integer sampling inherited from the non-MTP agentic siblings with the AgentX MTP grid: GPU-resident [1,4,8,12,16], offload [4,8,12,16]. Single-step sampling cannot separate configs by more than noise on the agentic corpus, and past conc 16 these SKUs are in the post-cliff thrashing regime. 中文：将沿用自非 MTP 智能体配方的逐 1 递增并发采样，替换为 AgentX MTP 并发网格： GPU 常驻 [1,4,8,12,16]，KV 卸载 [4,8,12,16]。相邻并发间隔至少为 2，且一律止于 16 —— 逐 1 采样在智能体语料上无法区分出超过噪声的差异，并发超过 16 后这些 SKU 已进入 显存瓶颈后的抖动区间。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
4d7e99f to
ed49211
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31207046020 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31207046020 |
# Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
…he coverage gate Run 31207046020 failed four cells, all TP8/EP8: GPU-resident c1 and c4, and Mooncake c4 and c8. None is an engine failure -- each served its requests with a 0% error rate, then AIPerf rejected the profile: ProfileMetricCoverageError: Profiling metric coverage below the required 98.0% for phase 'profiling': TTFT=95.0-97.6% At low concurrency on the long-context agentic corpus, too few sessions start inside the one-hour profiling window for TTFT to be sampled through to the end, so coverage lands just under the gate. The boundary is monotone and specific to the EP8 arms: pure TP8 passes at every concurrency including 1, while EP8 only clears the gate from conc 8 (GPU-resident) and conc 12 (Mooncake) upward -- which is also where expert parallelism begins doing useful work instead of paying all-to-all overhead on a nearly empty batch. Keeping those points would re-fail the sweep each run. GPU-resident EP8 [1, 4, 8, 12, 16] -> [8, 12, 16]; Mooncake EP8 [4, 8, 12, 16] -> [12, 16]. The TP8 arm is untouched. 14 configs down to 10. 中文：移除运行 31207046020 中失败的四个 TP8/EP8 点（显存驻留 c1、c4，Mooncake c4、 c8）。这些并非引擎故障——各点请求错误率均为 0%，失败于 AIPerf 的画像覆盖率门槛 （ProfileMetricCoverageError，TTFT 覆盖率 95.0%–97.6%，低于要求的 98.0%）：在长上下文 智能体语料下，低并发时一小时画像窗口内启动的会话过少，TTFT 无法采样至窗口末端。 该边界单调且仅影响 EP8 分支：纯 TP8 在包括并发 1 在内的所有点均通过，而 EP8 需分别 从并发 8（显存驻留）与并发 12（Mooncake）起才能通过——这也正是专家并行开始产生实际 收益、而非在近乎空批次上承担全互联开销的起点。保留这些点只会导致每次扫描重复失败。 显存驻留 EP8 由 [1, 4, 8, 12, 16] 调整为 [8, 12, 16]；Mooncake EP8 由 [4, 8, 12, 16] 调整为 [12, 16]；TP8 分支不变。配置数由 14 降至 10。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31237493975 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31237493975 |
1 participant
Summary
Adds
minimaxm3-fp8-h200-vllm-agentic-mtp— the EAGLE3 speculative-decoding (spec-decoding: mtp) variant of the MiniMax-M3 MXFP8 H200 AgentX recipe. One new benchmark script, one master-config entry, one changelog entry; no launcher change is needed (launch_h200-dgxc-slurm.shalready composesagentic/+_mtp).Speculative config —
method: eagle3onInferact/MiniMax-M3-EAGLE3,num_speculative_tokens: 3, drafter pinned toFLASH_ATTN(the EAGLE3 head is MHA; FlashInfer only serves the mandatory page size 128 via its trtllm-gen kernel, which needs GQA/MQA). Same draft and level as every merged MiniMax-M3 MTP recipe.Synthetic forced AL — throughput runs set
rejection_sample_method: syntheticwithsynthetic_acceptance_length: 2.83, the committed golden AL for MiniMax-M3 EAGLE3 at K=3,thinking_on(golden_al_distribution/minimaxm3_eagle3.yaml). Per the AgentX policy ingolden_al_distribution/README.md, a submission may choose any supported draft length but may not substitute a different acceptance target. The separateminimaxm3_eagle3_gqa.yamlcurve belongs to theInferact/MiniMax-M3-EAGLE3-GQAdraft (run 29784780049) and is not mixed in.EVAL_ONLYruns drop synthetic acceptance and keep real verification, since synthetic acceptance commits drafted tokens regardless of the target logits and would score the eval at ~0.Serve shape is otherwise identical to the non-MTP agentic sibling (Mooncake host-DRAM KV offload,
--block-size 128,--language-model-only,--kv-cache-dtype fp8,--attention-backend TRITON_ATTN,minimax_m3parsers, vllm-router for DP-attention) so the spec-decode delta is readable at equal concurrency.max-num-seqsstays2*CONCfor AgentX subagent fan-out; cudagraph capture is raised toMAX_NUM_SEQS*(1+3)tokens because vLLM's FULL_DECODE_ONLY ladder is token-denominated.Search space covers three arms (TP8 and TP8/EP8 GPU-resident, TP8/EP8 Mooncake) on the AgentX MTP concurrency grid: GPU-resident rows
[1, 4, 8, 12, 16], KV-offload rows[4, 8, 12, 16]. Every step is at least 2 concurrency apart and every arm stops hard at conc 16 — single-step sampling cannot separate configurations by more than run-to-run noise on the agentic corpus, and past 16 these SKUs are into the post-HBM-cliff thrashing regime the non-MTP sweeps already characterized. 14 generated jobs.Validation:
bash -nclean; both YAMLs parse;generate_sweep_configs.py test-configemits 14 agentic-coding configs withspec-decoding: mtpand the expected TP/EP/KV arms.中文说明
新增
minimaxm3-fp8-h200-vllm-agentic-mtp：MiniMax-M3 MXFP8 H200 AgentX（智能体编码）配方的 EAGLE3 投机解码（spec-decoding: mtp）变体。包含一个新的基准测试脚本、一条 master 配置条目和一条 changelog 条目；启动器无需改动（launch_h200-dgxc-slurm.sh已能拼出agentic/+_mtp路径）。投机解码配置：
method: eagle3，草稿头Inferact/MiniMax-M3-EAGLE3，num_speculative_tokens: 3，drafter 固定FLASH_ATTN（EAGLE3 头是 MHA，而 FlashInfer 仅通过要求 GQA/MQA 的 trtllm-gen kernel 支持强制的 page size 128）。与仓库中所有已合并的 MiniMax-M3 MTP 配方使用相同的草稿头与投机层数。合成强制 AL：吞吐量测试设置
rejection_sample_method: synthetic与synthetic_acceptance_length: 2.83，即 MiniMax-M3 EAGLE3 在 K=3、thinking_on 下已提交的黄金 AL（golden_al_distribution/minimaxm3_eagle3.yaml）。依据golden_al_distribution/README.md的 AgentX 规则，提交方可自选草稿长度，但不得替换接受长度目标。minimaxm3_eagle3_gqa.yaml对应的是Inferact/MiniMax-M3-EAGLE3-GQA草稿头（run 29784780049），两条曲线不混用。EVAL_ONLY场景关闭合成接受、保留真实验证，否则生成文本错误、评估分数接近 0。服务参数与非 MTP 版本完全一致（Mooncake 主机 DRAM KV 卸载、
--block-size 128、--language-model-only、--kv-cache-dtype fp8、--attention-backend TRITON_ATTN、minimax_m3解析器、DP-attention 下的 vllm-router），便于在相同并发下直接对比投机解码带来的差异。max-num-seqs保持2*CONC以容纳 AgentX 子智能体扇出；cudagraph 捕获上限提高到MAX_NUM_SEQS*(1+3)个 token，因为 vLLM 的 FULL_DECODE_ONLY 阶梯是按 token 计的。搜索空间覆盖三条分支（TP8 与 TP8/EP8 GPU 常驻、TP8/EP8 Mooncake），并采用 AgentX MTP 并发网格：GPU 常驻
[1, 4, 8, 12, 16]，KV 卸载[4, 8, 12, 16]。相邻并发间隔至少为 2，且一律止于并发 16 —— 在智能体语料上，逐 1 采样无法区分出超过运行间噪声的差异；而并发超过 16 后，这些 SKU 已进入非 MTP 扫描早已刻画过的显存瓶颈后抖动区间。共生成 14 个作业。验证：
bash -n通过；两个 YAML 均可解析；generate_sweep_configs.py test-config生成 14 条 agentic-coding 配置，spec-decoding: mtp与 TP/EP/KV 分支均符合预期。🤖 Generated with Claude Code
Revision (post-first-sweep)
--gpu-memory-utilization0.90. The first agentic job on [Klaud Cold] minimaxm3-fp8-h100-vllm-agentic-mtp: MiniMax-M3 H100 AgentX EAGLE3 recipe with golden-AL synthetic acceptance / MiniMax-M3 H100 AgentX EAGLE3 配方（黄金 AL 合成接受） #2423 (TP8/EP8 c9, run 30515793863) died mid-warmup: all 8 ranks raisedtorch.OutOfMemoryError96 MiB short, killing the engine core (EngineDeadError) and tripping AIPerf's terminal warmup failure. The recipe had inherited the non-MTP agentic sibling's gmu, which has no draft head to house; the proven MTP sibling on each of these SKUs runs 0.90 (explicit on H100/H200, vLLM's default on the ROCm ones). Applied to all four MiniMax PRs.中文：(1)
--gpu-memory-utilization改为 0.90 —— #2423 的首个智能体作业（TP8/EP8 c9，run 30515793863）在预热阶段 8 个 rank 全部 OOM（仅差 96 MiB），导致 engine core 崩溃并触发 AIPerf 终止性预热失败；此前沿用了非 MTP 版本的取值，而该版本无需为草稿头预留显存，各 SKU 上已验证的 MTP 配置均为 0.90。四个 MiniMax PR 均已修正。(2) 并发网格：相邻间隔至少为 2，且一律止于 16。