Add NVFP4 MiniMax-M3 4xRTX PRO 6000 recipe - #678
Conversation
Signed-off-by: Oseltamivir <58582368+Oseltamivir@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the MiniMax-M3.yaml model configuration to support and document the RTX PRO 6000 GPU configuration for the NVFP4 variant. This includes increasing the minimum VRAM requirement to 301 GB, adding a new deployment guide section for RTX PRO 6000 TP4 + EAGLE3, and referencing the necessary vLLM PRs and validation runs. Feedback on the changes points out that the RTX PRO 6000 is based on the Ada Lovelace architecture and does not support native FP4 tensor cores, suggesting a clarification in the description to note that FP4 execution is emulated on Ada GPUs.
| description: "NVIDIA-quantized NVFP4 weights (ModelOpt) for Blackwell (B200/B300) — native FP4 tensor cores, ~half the VRAM of MXFP8." | ||
| # The mixed-precision checkpoint is 250.1 GB on disk; allow 20% headroom. | ||
| vram_minimum_gb: 301 | ||
| description: "NVIDIA-quantized NVFP4 weights (ModelOpt) for Blackwell (B200/B300 and RTX PRO 6000) — native FP4 tensor cores." |
There was a problem hiding this comment.
The description states that the NVFP4 variant is for "Blackwell (B200/B300 and RTX PRO 6000) — native FP4 tensor cores." However, the RTX PRO 6000 is based on the Ada Lovelace architecture (compute capability 8.9) and does not have native FP4 tensor cores. Native FP4 tensor cores are a Blackwell-exclusive feature. On Ada Lovelace GPUs like the RTX PRO 6000, FP4 execution is emulated or run via dequantization (e.g., Marlin). Please clarify this in the description to avoid misleading users about the hardware capabilities of the RTX PRO 6000.
description: "NVIDIA-quantized NVFP4 weights (ModelOpt) for Blackwell (B200/B300) and RTX PRO 6000 — native FP4 tensor cores on Blackwell, emulated on Ada."
Oseltamivir
changed the title
1 participant
Adds a validated 4× RTX PRO 6000 launch recipe for MiniMax-M3 NVFP4 with TP4/TP4+EP4 and EAGLE3. It also corrects the checkpoint-based NVFP4 VRAM estimate.
Validation: InferenceX run 30066976722.
Merge dependency: vLLM #48929 must land first.