Repository navigation
Ignore unreferenced Inductor kernels in static launcher bundles - #185
Merged
lukealonso merged 1 commit intoNov 4, 2026
Merged
Conversation
Inductor's epilogue-fusion benchmarking compiles candidate template kernels that keep the unrenamed Placeholder.DESCRIPTIVE_NAME. With the static launcher enabled they enter the forward's static autotuner bundle even when final code regenerates the chosen fusion under a different source key. Exact bundle equality in InductorSelections.register then rejected the forward, so qwen3.8-27b failed to warm speculative32x8p128 on SM120. Registration still requires a bundled autotuner for every kernel the generated module assigns, and still rejects an extra entry whose kernel name appears in the generated source. Only entries the module never names are ignored. Freezing already rebuilds the saved bundle from bound kernels, so frozen forwards and packages are unchanged. Validation on RTX PRO 6000 Blackwell Max-Q (SM120), torch 2.14.1+cu132: - New single-GPU test adds an unreferenced candidate to a compiled forward's bundle; registration succeeds, the restored frozen forward omits it and neither compiles nor benchmarks, and graph replay matches. A named unbound entry still raises. The test fails on cc696a6 with the original error. - 65 CPU tests in the forward compile, artifact sharing, prepared winner, Triton serde, and restored launch suites passed. - test_inductor_tuning_cuda.py single-GPU: 5 passed. Three existing tests failed because this container cannot lock GPU clocks; the two-rank tests were not run. - qwen3.8-27b export (warm caches from a failing cc696a6 run, two attempts) now registers and warms speculative32x8p128 and continues to forwards 51 and 53 of 108. Limitations: the 27B export still does not complete. During warmup, device memory outside the PyTorch allocator grows by about 1 GB per warmed forward (42.8 GB at warm start to the 95 GB limit), and warmup fails with CUDA out of memory. That separate defect has not been diagnosed, so no 27B package was exported or served. A standalone forward with the selector's bmm and epilogue did not reproduce the orphaned entry, so the test supplies one directly. Muse Spark and TP > 1 exports were not tested. Addresses NVIDIA#183. Design-Signed-Off-By: Cursor Agent (Auto); design-review=1c01
This was referenced Nov 3, 2026
mark-yong
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ignition export qwen3.8-27b --gpus 0fails onmainwhile warmingspeculative32x8p128:compiled forward has unbound static Inductor kernels(#183). The forward's static launcher bundle holds one extra entry: a fused-epilogue variant of the DFlash2 candidate selector'sbmmtemplate that the generated module never references. Its name is still Inductor'sPlaceholder.DESCRIPTIVE_NAME, which suggests that epilogue-fusion benchmarking compiled it.InductorSelections.registerrequired exact bundle equality, so it rejected the forward.Change
registerstill requires a bundled autotuner for every kernel the module assigns, and still rejects an extra entry whose kernel name appears in the generated source. It now ignores extra entries the module never references.freezealready rebuilds the saved bundle from bound kernels, so frozen forwards and packages are unchanged.Validation
RTX PRO 6000 Blackwell Max-Q (SM120), torch 2.14.1+cu132:
cc696a6b.test_inductor_tuning_cuda.py, 5 single-GPU tests pass. Three failed only because the container cannot lock GPU clocks, and the two-rank tests were not run.qwen3.8-27bexport now warmsspeculative32x8p128and continues to forward 51–53 of 108.Limitations
Fixes #183.