Skip to content

Windows: prepare, export and serve prepared packages on RTX Spark N1X - #159

Merged
lukealonso merged 1 commit into
NVIDIA:mainfrom
R4yHarris:windows-n1x-export
Nov 2, 2026
Merged

lukealonso merged 1 commit into
NVIDIA:mainfrom
R4yHarris:windows-n1x-export

Conversation

@R4yHarris

@R4yHarris R4yHarris commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Preparing, exporting and serving a prepared package on ARM64 Windows failed at several independent points, each with its own error:

  • cuMemcpyDtoHAsync_v2: CUDA 1: invalid argument — a recorder-discovered allocation (66 MiB) had been released before capture; its address was reused and its extent only partially mapped, so the storage scan aborted the export.
  • retained device allocation "external-0" … below the restorable address floor 0x4000000000 — the storage floor was asserted even when the low window had not been sealed (WDDM owns it; cuMemAddressReserve cannot constrain cuMemAlloc).
  • NVML is required to name an export: LoadLibraryExW failed and NVML reports unsupported GPU PCI id 0x2e03 — the loader asked for libnvidia-ml.so.1, and the PCI table only knew DGX Spark's 0x2e12.
  • Access is denied (os error 5) from File::open(path).sync_all() — FlushFileBuffers needs write access, and published files are sealed read-only.
  • LockFileEx cannot change a lock's type in place; lock contention arrives as ERROR_LOCK_VIOLATION (33), not WouldBlock.
  • package path escapes its root — fs::canonicalize yields a \\?\ verbatim path while the opened-file path is stripped.
  • direct I/O sector size 32768 does not divide 4096 — the check read PhysicalBytesPerSectorForPerformance, a hint.
  • ImportError: cannot import name 'DupFd' — peer_storage imported a POSIX-only symbol at module level, and the compile-pool pickler imports that module, so every default-effort export died right after tuning started.
  • Loading a package failed reading the tokenizer: an empty relative payload name mapped to the entry directory instead of the primary container.
  • cannot find function 'strftime' in crate 'libc' — the Jinja strftime_now frontend calls libc::strftime, which the crate does not declare for Windows, so helios did not compile.
  • cannot find 'UnixStream' in 'net' — the selene crate matched PoolAddress::Unix with tokio::net::UnixStream, which does not exist on Windows, so helios and heliopause did not compile at all.
  • Preparation patches require the preparation dependencies — the patch stage built its package map eagerly, so distribution("triton") raised PackageNotFoundError. torch depends on triton only on Linux (uv.lock marks it sys_platform == 'linux') and triton publishes no Windows wheel, so every export on this platform stopped just after GPU selection.
  • A panic reading Kimi copyright header at helios/src/frontend/templates.rs:54 — the Kimi chat template is embedded with include_str! and parsed with split_once("#}\n"), but a core.autocrlf=true checkout stores that asset with CRLF, so the split cannot match and serving panicked at Load tokenizer. The same checkout also moves the bytes that the Qwen and Step templates' result_sha256 digests were computed over.

Resulting behavior

  1. export: tolerate recorder allocations released during preparation — the recorder tracks the owning pool of cuMemAllocFromPoolAsync and drops a pool's allocations on cuMemPoolDestroy; Reachability::forget() drops a record so a later pointer hit cannot re-reach it; capture distinguishes recorder-discovered allocations, reads each discovered extent end to end, omits the ones that are no longer readable, and fails loudly if a graph-referenced record is unreachable. Copy failures name the allocation, address, offset and size.
  2. package cache: commit and publish entries on Windows — sync_file(), lock downgrade by unlock-then-relock, contention() accepting ERROR_LOCK_VIOLATION, comparable_root() stripping the verbatim prefix, named commit-lock failures, and a rewrite helper that clears a sealed destination's read-only attribute before renaming.
  3. naming: recognise the RTX Spark N1X PCI device id — GB10_DEVICES = [0x2e12, 0x2e03].
  4. runtime_package: read the primary container for an empty relative name.
  5. eos: accept volumes whose performance sector hint exceeds the logical sector.
  6. preparation: run preparation and export on Windows — storage reports whether it sealed and exposes address_floor accordingly; peer_storage guards the POSIX-only import and raises a clear error for the operation.
  7. selene: reject Unix-domain manager sockets where they cannot be opened — the Unix arm is gated to Unix platforms and reports the address as unreachable elsewhere. PoolAddress keeps the variant so manifests and tests that name it stay portable; only the transport that cannot exist is refused.
  8. frontend: declare strftime for the Windows C runtime — MSVC's CRT exports strftime; the Windows declaration is unsafe extern per edition 2024 and the call sits in an explicit unsafe block. Other platforms keep libc's binding.
  9. patches: apply the pinned fixes to the packages the platform installs — the patch stage collects only installed packages, keeps the exact-version requirement for each of them, and skips a bundled patch whose target package is absent. A package the platform cannot install has no files to modify, so its patch has nothing to apply; the version guarantee is unchanged for everything present.
  10. assets: keep chat template line endings stable across checkouts — the template assets are declared text eol=lf, so every checkout produces the bytes the template digests were computed over.
  11. tests: make the helios test profile compile on Windows — the flare-selene-manager dev-dependency and its single consumer (offload::pool::tests) are Unix-scoped, since that crate binds Unix-domain sockets and calls flare_eos::direct_update_file, which exists only in eos/src/unix.rs; the MemAvailable parser is compiled on every platform because cfg(test) never reaches a dependency; and two cache fixtures use a hyphen instead of a colon, which Windows cannot represent as a regular file name.
  12. package: serialise package mutation on Windows — lock_package took a lock only under cfg(unix) and did nothing on Windows, so two concurrent emit_derivations calls both rewrote the package and one derivation was lost. The Windows path now takes a blocking exclusive LockFileEx.
  13. selene: fill the record token from the Windows CSPRNG — shard::nonce read /dev/urandom on every platform, so a two-rank join failed startup on Windows with os error 3 and the peer saw a connection reset. Windows now uses BCryptGenRandom with the system-preferred RNG.
  14. tests: clear the sealed attribute before corrupting a cached payload — the corrupted-bytes warm-hit test relaxed a Unix file mode and could not write the read-only payload on Windows.

Validation

  • A full-capacity export on RTX Spark N1X (ARM64 Windows, GB10 die, TP1) completed every phase and published a package: 112 captured graphs, concurrency 32, image concurrency 4, five MTP draft tokens, 262,144-token context, 4,096-token prefill chunks. The log records omitting released allocation 0xb03800000+692****6016 recorded during preparation.
  • cargo test package::cache (x86_64-pc-windows-msvc, --features runtime-package): 76 passed, 0 failed, 1 ignored (previously 47 passed / 25 failed). Before commit 11 the test profile did not compile on Windows at all, so no helios test could run.
  • Full helios library suite on x86_64-pc-windows-msvc: 791 passed, 1 failed, 19 ignored. The one failure is the rename-versus-immutability conflict in the limitations below; it was the only failure left after commits 11–14.
  • cargo test for eos: 5 passed. package::naming: 6 passed on x64 and on the native aarch64 host.
  • runtime_package: 108 passed, 1 failed — see limitations.
  • The published package loads and serves: /v1/models responds, eight validation prompts were answered correctly (arithmetic, instruction following, multi-turn, code, JSON, time arithmetic, one-word edge case, ~4k-token needle), and two concurrent requests completed cleanly.
  • The package converts to the packed form (106,927,156,224 bytes) through the rebuilt extension, and serves agent turns through an OpenAI-compatible client.
  • With commits 7–9 applied, heliopause builds for Windows ARM64 and flare builds in release (2m10s), where neither compiled before.
  • ignition.patches.apply() on Windows ARM64 patches torch/_inductor's triton_heuristics.py and static_triton_launcher.py and skips the triton patch; it previously raised PreparationError before preparation began.
  • On the updated tree an export reaches and completes kernel tuning, weight loading, embedding-table preparation (26.8 GiB) and graph catalog construction (112/112 in 148 s), where the patch-stage failure previously stopped it after GPU selection.
  • With the template asset renormalized and flare rebuilt, the ABI 5 package for Qwen3.8-Flash-Next loads past Load tokenizer (248,320-token vocabulary) and serves: eight of eight validation prompts correct, two concurrent requests completed, median decode 31.4 tok/s over the token window (0.27–0.85 s to first token). Before the change the same package panicked at that phase. An earlier figure of 309.4 tok/s came from timing TTFT on the first content chunk while this checkpoint streams reasoning first, which shrinks the decode window to the tail of generation; the corrected harness counts both channels.

Sign-off

All 14 commits carry the DCO sign-off required by CONTRIBUTING.md: Signed-off-by: R4yharris <harray86@me.com>.

Material limitations

  • authenticated_handles_survive_rename_and_replacement still fails. eos::windows::direct_read_guard deliberately reopens a package with FILE_SHARE_READ only to keep it immutable, which prevents renaming the directory the test renames. Narrowing that sharing was tried and reverted (it broke concurrent writers with a sharing violation), so this is left as a maintainer decision on which guarantee wins rather than patched here.
  • Validated on one GPU with TP1 only; no multi-GPU or cross-node evidence. One NVMe and one Windows build.
  • Windows branches were exercised; the Unix branches are unchanged apart from routing through sync_file().
  • The triton patches never apply on Windows, because triton cannot be installed there. torch and FlashInfer JIT compilation are unaffected: the Windows preparation path compiles with nvcc and does not load triton.
  • No Windows path reaches a Unix-domain manager socket; the refusal in commit 7 is the whole Windows behavior for that address. selene-manager itself is not ported, so the offload pool tests stay Unix-only and Windows has no coverage of those paths.
  • The Windows strftime passes the format to the CRT, so it accepts the CRT's directives rather than POSIX extensions the platform lacks.
  • Existing clones need one git add --renormalize and a re-checkout of the template asset before the line-ending rule takes effect.

@R4yHarris
R4yHarris force-pushed the windows-n1x-export branch 3 times, most recently from 84a2cbe to 75bca4b Compare October 31, 2026 17:03
@R4yHarris

Copy link
Copy Markdown
Contributor Author

Thanks for taking a look. Two items need a maintainer:

  1. The Native serving workflow for this branch is sitting in action_required (fork PR), so CI has not run on either matrix leg. Could someone approve the run?

  2. authenticated_handles_survive_rename_and_replacement fails on Windows by a design conflict rather than a bug: eos::windows::direct_read_guard opens a package with FILE_SHARE_READ only to keep it immutable, which blocks the directory rename the test performs. Narrowing that sharing was tried and reverted because it broke concurrent writers with a sharing violation. I have left this as your call on which guarantee wins rather than patching around it, and I am happy to implement whichever you prefer.

All commits carry Signed-off-by per CONTRIBUTING.md, and the full Windows library suite is 791 passed / 1 failed / 19 ignored, that one failure being the item above.

@lukealonso

Copy link
Copy Markdown
Collaborator

FWIW - you can build the model package on Linux and it'll work fine on Windows, but of course it should also work on Windows directly. I'll take a look at this. Thanks!

Use the Windows NVML library name, recognize the N1X GB10 PCI ID, and
resolve primary-container asset paths when the relative filename is empty.
Check direct-I/O geometry against logical and atomic sector sizes, preserve
LF chat-template bytes, and generate peer tokens with BCryptGenRandom.

Permit importing peer storage without POSIX descriptor passing, with an
explicit error when that operation is requested. Allow absent Triton only
on Windows while retaining required PyTorch and exact installed versions.
Cover directory and packed asset reads and dependency validation.

Validation: 364 package tests and one shard handshake test passed on Linux
(7 ignored); 38 preparation tests passed (1 skipped). The Windows x64
Helios test binary cross-compiled, and Heliopause passed cargo check.
Template bytes were preserved with core.autocrlf=true, and peer storage
imported with DupFd unavailable. Native Windows execution and GPU export
and serving were not exercised.

Signed-off-by: R4yharris <harray86@me.com>
Design-Signed-Off-By: Codex (GPT-6); design-review=1c01
Signed-off-by: Luke Alonso <lalonso@gmail.com>
@lukealonso
lukealonso merged commit 42481f9 into NVIDIA:main Nov 2, 2026
original-el8 added a commit that referenced this pull request Nov 3, 2026
main carries the RTX Spark N1X serving fixes from #159 that this branch
cherry-picked. eos/src/windows.rs conflicted only in the comment above the
direct-I/O sector check; the code is identical on both sides, and main's
comment is kept. The branch now differs from main only by the commit bound
on Windows integrated-GPU cache sizing and its documentation.

Validation on an RTX Spark N1X, Windows 11 ARM64, aarch64-pc-windows-msvc:
flare builds; helios runtime_package::memory 13 passed,
runtime_package::protection 15 passed, package::naming 6 passed; eos 5
passed.

Design-Signed-Off-By: Claude Code (Claude Opus 5.5); design-review=1c01
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Jason Cook <1637905+original-el8@users.noreply.github.com>
R4yHarris added a commit to R4yHarris/spark-local-inference that referenced this pull request Nov 5, 2026
…dows RTX Spark N1X Export & Serving in docs/flare-fork-pr-159-review.html
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants