Repository navigation
GB10 TP2: kernel tuning MemoryError with MTP (glm-5.3-flash); vm.min_free_kbytes hides ~6.6 GB from the headroom check #73
Description
Setup: 2x DGX Spark (GB10, 128 GB unified each), direct CX7 RoCE link, Flare main @d5fd9824 (scripts/build_deb.sh), Orbit + flare-agent TP2. Model nvidia/GLM-5.3-Flash-FP4-Local@76348dd, --max-concurrency 8, --prefill-chunk-size 2048. Stock DGX OS, driver 580.178.04.
Problem: the MTP export fails in kernel tuning with MemoryError on both ranks. Tuning finds 7.2-8.8 GiB free and needs 9,216 MiB (8 GiB headroom + 1 GiB growth), measured after scratch reservation, which takes about 9.5 GiB whatever the concurrency. The same export without MTP passes.
Didn't help: dropping page cache, --compile-workers 4, chunk 2048, lower speculative capacity or image concurrency, c4.
Workaround: our boxes run vm.min_free_kbytes=4194304, a common Spark setting against GPU-driver OOM freezes. On GB10 the kernel watermarks hide about 6.6 GB per box from cudaMemGetInfo. Lowering it to 2 GiB for the export only gave back about 3.7 GB and the MTP export passed. Serving then works with 4 GiB restored: KV 1,993,728 tokens (6.9 GiB per rank), 1M tokens per request, c8.
Asks:
- On unified-memory parts, could the tuning headroom be configurable or scale with total memory?
- Could the
MemoryErrorprint the free and needed numbers, and mentionmin_free_kbyteson GB10?
Result once running, for reference:
- tools 5/5
- our 8-case image test 8/8; the tiny-solid-colour-after-20K case also came back right
- Hotel Lights (llm-inference-bench) xhigh 8/8
- decode 41 / 81 / 121 tok/s at C1 / C4 / C8
- prefill 1,683-2,018 tok/s from 8K to 128K
So GLM + MTP + vision on 2x GB10 TP2 works.
Minor, same build: at d5fd982, packaging/debian/Dockerfile and Dockerfile.dockerignore don't COPY or allow the recipe assets for gemma4, glm53, kimi3, mimo_v2, minimax_m3, nemotron_h, qwen38 and step5. build_deb.sh failed on a missing kimi3 template until we added them.
Activity
javieratomic commentedon Oct 28, 2026
Some numbers from a 3x GB10 setup (DeepSeek V4.1 Flash, TP3) in support of making the headroom configurable.
On GB10 the fixed 8 GiB
_HEADROOM_BYTESinignition/preparation/host_memory.pyis what sizes the compiler pools once the weights are resident. During an export each node has about 10-13 GiB available, which leaves one forward compiler per rank. For a local experiment only, I patched the constant to 6 GiB. I know AGENTS.md rules out environment switches, so this is not a proposal for the mechanism.I then timed "Compile model forwards" on its own: tuning was restored from cache, forwards started cold, and everything else was identical.
--compile-workers 4, headroom 8 GiBWith 6 GiB the lowest MemAvailable on any node during the phase was 9 GiB, and there was no OOM. A full export with 6 GiB of headroom completed and the package served normally: at 262k tokens, the cold prefill took 77.6 s and the warm one 1.8 s, retrieval scored 16/16, and decode ran at 42.7 tok/s with one stream and 93.5 tok/s aggregate with four. All of these numbers are in line with the 8 GiB exports.
--compile-workers Nis slower than the default because it forcescompile_threads=1(compile_pool.py), while the memory budget still caps the pool at one worker. That is easy to misread as "more workers".Two small suggestions:
PreparationConfig, so the report records it, and keep 8 GiB as the default. Make the compiler memory headroom a preparation setting #122 does this as[preparation] compile_headroomand--compile-headroom. With either one set to 6G, "Compile model forwards" took 508–511 s, against 906 s with the default.all_gatheruntil the end of the phase before it raises theMemoryError.CaptKitCarson commentedon Nov 2, 2026
Update for Qwen, for anyone hitting this on GB10: at
6fa01a49a Qwen3.8 Flash Next TP2 export with MTP 5 and c8 passed kernel tuning on the stockvm.min_free_kbytes=4194304, with no change to the setting ("Prepare and tune kernels" 788.1 s, FlashInferf5f5341d).This isn't a retest of the GLM-5.3-Flash MTP export this issue was opened for, and I haven't checked whether the headroom check itself changed, so it doesn't show the issue is fixed.
feat(#10): Fix(ignition): include vm.min_free_kbytes hint in host_mem…