[voipmonitor]

Collective startup tuning captures each reduction group separately, so a group's samples run consecutively instead of being separated by other prepared groups. To measure candidates across that broader execution sequence, the groups must share a startup graph while retaining independent selectors and failure checks.

This branch replaces per-group startup graphs with one graph that interleaves 21 sample passes across all required groups. Ignition captures it in the package graph pool, retains each group's selector and status, and publishes an aggregate status through caller-owned storage. Native loading keeps its zero-success/nonzero-failure contract; diagnostics count selector bindings. Reduced serving capacities retain the aggregate graph and its workspace. The ladder reader also admits destination-serialized profiles subject to FlashInfer's rank and CTA constraints. These changes depend on FlashInfer #62, pinned at 729464839, for the interleaved producer and serialized writers; that dependency must be resolved before this integration can merge. No native operation contract or shutdown deadline changes.

The expected benefit is less dependence on consecutive samples of one collective group. A serving-throughput improvement is not qualified, and this PR remains research-only for performance. The companion kernel has a measured switched-fabric regression. FlashInfer #62 also displays a broader, unqualified comparison against its rewritten main history; the focused integration results below do not qualify that entire dependency diff.

Correctness qualification at Flare c2310615, incorporating 90b5c6d3, with FlashInfer 729464839: 169 focused CPU checks and six GPU cases passed. Automatic export checks startup replay and device allocations within the package address window. Native TP4 writer fixtures passed on host-bridge-connected GPUs 8/10/12/14 and switched GPUs 12–15. Protocol counters and generation-tagged queue completion verify that the selected writer executes. Checks cover bytewise arithmetic, immutable selectors, allocation stability, peer-import reachability, retained owners, and native drain.

The performance blocker is the companion payload_rsag kernel, which performs payload reduce-scatter/all-gather. On switched RTX PRO 6000 SM120 TP4, BF16 [31,5120], 512 threads and 64 active CTAs, sustained probes measured 12.244 → 14.552 µs p50 (+18.9%). The affected range is approximately 17–32 rows. Cases used at least two seconds of warmup and three seconds of timing with repeated upstream controls. The requested graphics clock was 3090 MHz; observed SM clocks were 2842–2910 MHz. The cause remains unresolved.

The following whole-model measurements are upstream controls, not before/after results for this PR. Both runtime packages used FlashInfer 0b6178857:

DeepSeek V4.1 Flash, RTX PRO 6000 SM120 TP4	C1 tok/s	C8 tok/s	C16 tok/s
Flare eb71702b, host bridges, GPUs 0/4/8/12	190.35	345.74	391.53
Native-static host-bridge reference	247.90	463.16	511.29
Flare eb71702b, switched, GPUs 12–15	319.29	869.97	1162.69
Flare 90b5c6d3, host bridges, GPUs 0/4/8/12	163.47	385.11	348.20
Flare 90b5c6d3, switched, GPUs 12–15	309.59	855.32	1168.18

Conditions: prepared concurrency 16, fixed verification width 8, 64G prefix cache, two 30-second cells per concurrency, and unmodified boost clocks. Every row, including the static reference, used Engram weights on disk, scales in RAM, and a 4 GiB row cache. RAM-delivery comparisons are unqualified. Packages gb202-tp4-202610290017 and gb202-tp4-202610291301 retained different complete kernel catalogs, so their difference cannot isolate one preparation change.

For those two packages respectively, 81 startup groups took 6.96/6.66 s on host bridges and 4.12/3.74 s switched. Package 0017 measured host-bridge C16 GPU intervals of 50.88/52.01 ms per round versus 39.62 ms for the static reference. Preparation resource release took 30.5/15.3 s, both within the unchanged 60 s deadline; package 1301 completed one full garbage collection per rank. Those release results qualify the upstream reclamation implementation, not a lifecycle change in this PR.

Remaining gates are the companion regression, measured ladder additions, final native replay, full-model C1/C8/C16 qualification, three independent starts, and two independent exports. No throughput target is claimed as met.

[original-el8]

Marked triage:design-review: the remaining proposal is interleaved collective startup sampling and PCIe profile selection. The PR itself confirms that tuning-owner reclamation is already on main, so that resolved lifecycle problem should not justify merging the remaining changes.

The proposed FlashInfer dependency PR #62 is still open. Resolve the startup-selection design and dependency together, then remeasure against main's startup-storage changes from #197. This is not a closure candidate solely because the reclamation work is complete.

[original-el8]

@voipmonitor GitHub reports that this PR has merge conflicts with main. A local git merge-tree check also confirms conflicts for PR head c231061573e6 against main at c65024dba32b (2026-11-05).

Conflicting files:

• helios/src/execution/tests.rs
• helios/src/execution/transport.rs
• ignition/preparation/collective_ladder.py
• ignition/preparation/ladders/README.md
• ignition/preparation/startup_tuning.py
• pyproject.toml
• tests/preparation/test_collective_ladder.py
• tests/preparation/test_startup_tuning.py
• uv.lock

Please update the branch against main (rebase or merge), resolve these conflicts, and rerun the affected checks.