[voipmonitor]

Cold startup races read 256 MiB only to displace L2. Resident weights serve the same purpose.

Changes

File	Change
ignition/preparation/startup_tuning.py	before the first capture: largest resident checkpoint storage → b12x.comm.pcie.use_startup_eviction
pyproject.toml, uv.lock	FlashInfer → voipmonitor/flashinfer@f2868d9a (voipmonitor/flashinfer#6)
tests/preparation/test_startup_tuning.py	eviction source named before the first capture (new)

Effect: DS4.1 Flash TP4 gb202, per GPU

	gb202	POC 2
Contents	base	this PR + flashinfer PR #6 + descriptions PR
L2 eviction storage	5 × 256 MiB	0 (reads weights)
Loaded at serving	82.25 GiB	79.57 GiB
KV cache budget	12.16 GiB	14.84 GiB
KV tokens	13,777,024	17,010,176
C32 GPU ms/step	28.98	28.94
Tetris steps/s	129.08	129.15

• Per-change breakdown: flashinfer PR Prepare clustered PCIe collectives with execution ABI 5 #6.
• Reference without runtime tuning: 17,847,552 KV tokens.

Requires

• b12x pcie: share startup race storage (DS4.1 gb202 KV 13.8M → 17.0M) flashinfer#6 (use_startup_eviction; older FlashInfer fails on import)

Compatibility

• package format: unchanged
• runtime: unchanged; the POC 2 package loads in flare:release-18b16e1-main-20261104 with the same KV
• races only read the weights; values are irrelevant

Reproduce (gb202 flags)

uv sync --locked --extra prepare --extra test
.venv/bin/ignition export deepseek-v4.1-flash \
  --gpus 0,1,2,3 --max-concurrency 32 --image-concurrency 4 \
  --mtp-draft-tokens 7 --speculative-token-capacity 128 --speculative-graph-budget 64 \
  --prefill-chunk-size 4096 --mla-compute-mode bf16 --expert-scale-compression native \
  --autotune-effort default --compile-workers 64 \
  --collective-tuning runtime \
  --collective-ladder ignition/preparation/ladders/gb202_pcie_tp4_clustered_full.json \
  --draft-head-quantization nvfp4 --markov-head-quantization fp8 \
  --vocabulary-parallel-sampling

flare run <package>/model.safetensors --gpus 0,1,2,3 --max-concurrency 32 \
  --mtp-draft-tokens 7 --mtp-adaptive-verification --mtp-max-verification-token-budget 128 \
  --mtp-adaptive-verification-alpha 0.6 --scheduler-policy balanced \
  --embedding-table-storage ram --prefix-cache-memory 64G --kv-cache-limit memory

• Check in the startup log: Allocate cache. … KV tokens.

Tests

Test	Result
tests/preparation/test_startup_tuning.py + tests/direct/test_executor.py	30 pass
merged with the descriptions PR (clean merge)	31 pass
Full model POC 2	KV 17,010,176; smoke, 4 parallel images: pass
Startup choices vs gb202	110/111 groups equal

Artifacts (lab)

	Path
POC 2 package	/data/ssd2/claude-static-export/cache/models/deepseek-ai_2FDeepSeek-V4.1-Flash-1017/gb202-tp4-202611041244
POC image (both Flare PRs)	claude-local/flare:kvpoc-10943f8e
Export and serving logs	/data/ssd2/claude-kvpoc/logs/

Related

• Named startup choices, /v1/flare/collectives: Name startup collective choices; serve them at /v1/flare/collectives #8

🤖 Generated with Claude Code