Repository navigation
Windows: prepare, export and serve prepared packages on RTX Spark N1X - #159
Conversation
84a2cbe to
75bca4b
Compare
|
Thanks for taking a look. Two items need a maintainer:
All commits carry |
|
FWIW - you can build the model package on Linux and it'll work fine on Windows, but of course it should also work on Windows directly. I'll take a look at this. Thanks! |
Use the Windows NVML library name, recognize the N1X GB10 PCI ID, and resolve primary-container asset paths when the relative filename is empty. Check direct-I/O geometry against logical and atomic sector sizes, preserve LF chat-template bytes, and generate peer tokens with BCryptGenRandom. Permit importing peer storage without POSIX descriptor passing, with an explicit error when that operation is requested. Allow absent Triton only on Windows while retaining required PyTorch and exact installed versions. Cover directory and packed asset reads and dependency validation. Validation: 364 package tests and one shard handshake test passed on Linux (7 ignored); 38 preparation tests passed (1 skipped). The Windows x64 Helios test binary cross-compiled, and Heliopause passed cargo check. Template bytes were preserved with core.autocrlf=true, and peer storage imported with DupFd unavailable. Native Windows execution and GPU export and serving were not exercised. Signed-off-by: R4yharris <harray86@me.com> Design-Signed-Off-By: Codex (GPT-6); design-review=1c01 Signed-off-by: Luke Alonso <lalonso@gmail.com>
75bca4b to
e88aba7
Compare
main carries the RTX Spark N1X serving fixes from #159 that this branch cherry-picked. eos/src/windows.rs conflicted only in the comment above the direct-I/O sector check; the code is identical on both sides, and main's comment is kept. The branch now differs from main only by the commit bound on Windows integrated-GPU cache sizing and its documentation. Validation on an RTX Spark N1X, Windows 11 ARM64, aarch64-pc-windows-msvc: flare builds; helios runtime_package::memory 13 passed, runtime_package::protection 15 passed, package::naming 6 passed; eos 5 passed. Design-Signed-Off-By: Claude Code (Claude Opus 5.5); design-review=1c01 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Jason Cook <1637905+original-el8@users.noreply.github.com>
…dows RTX Spark N1X Export & Serving in docs/flare-fork-pr-159-review.html
Problem
Preparing, exporting and serving a prepared package on ARM64 Windows failed at several independent points, each with its own error:
cuMemcpyDtoHAsync_v2: CUDA 1: invalid argument— a recorder-discovered allocation (66 MiB) had been released before capture; its address was reused and its extent only partially mapped, so the storage scan aborted the export.retained device allocation "external-0" … below the restorable address floor 0x4000000000— the storage floor was asserted even when the low window had not been sealed (WDDM owns it;cuMemAddressReservecannot constraincuMemAlloc).NVML is required to name an export: LoadLibraryExW failedandNVML reports unsupported GPU PCI id 0x2e03— the loader asked forlibnvidia-ml.so.1, and the PCI table only knew DGX Spark's0x2e12.Access is denied (os error 5)fromFile::open(path).sync_all()—FlushFileBuffersneeds write access, and published files are sealed read-only.LockFileExcannot change a lock's type in place; lock contention arrives asERROR_LOCK_VIOLATION(33), notWouldBlock.package path escapes its root—fs::canonicalizeyields a\\?\verbatim path while the opened-file path is stripped.direct I/O sector size 32768 does not divide 4096— the check readPhysicalBytesPerSectorForPerformance, a hint.ImportError: cannot import name 'DupFd'—peer_storageimported a POSIX-only symbol at module level, and the compile-pool pickler imports that module, so every default-effort export died right after tuning started.cannot find function 'strftime' in crate 'libc'— the Jinjastrftime_nowfrontend callslibc::strftime, which the crate does not declare for Windows, soheliosdid not compile.cannot find 'UnixStream' in 'net'— theselenecrate matchedPoolAddress::Unixwithtokio::net::UnixStream, which does not exist on Windows, soheliosandheliopausedid not compile at all.Preparation patches require the preparation dependencies— the patch stage built its package map eagerly, sodistribution("triton")raisedPackageNotFoundError.torchdepends on triton only on Linux (uv.lockmarks itsys_platform == 'linux') and triton publishes no Windows wheel, so every export on this platform stopped just after GPU selection.Kimi copyright headerathelios/src/frontend/templates.rs:54— the Kimi chat template is embedded withinclude_str!and parsed withsplit_once("#}\n"), but acore.autocrlf=truecheckout stores that asset with CRLF, so the split cannot match and serving panicked at Load tokenizer. The same checkout also moves the bytes that the Qwen and Step templates'result_sha256digests were computed over.Resulting behavior
export: tolerate recorder allocations released during preparation— the recorder tracks the owning pool ofcuMemAllocFromPoolAsyncand drops a pool's allocations oncuMemPoolDestroy;Reachability::forget()drops a record so a later pointer hit cannot re-reach it; capture distinguishes recorder-discovered allocations, reads each discovered extent end to end, omits the ones that are no longer readable, and fails loudly if a graph-referenced record is unreachable. Copy failures name the allocation, address, offset and size.package cache: commit and publish entries on Windows—sync_file(), lock downgrade by unlock-then-relock,contention()acceptingERROR_LOCK_VIOLATION,comparable_root()stripping the verbatim prefix, named commit-lock failures, and a rewrite helper that clears a sealed destination's read-only attribute before renaming.naming: recognise the RTX Spark N1X PCI device id—GB10_DEVICES = [0x2e12, 0x2e03].runtime_package: read the primary container for an empty relative name.eos: accept volumes whose performance sector hint exceeds the logical sector.preparation: run preparation and export on Windows— storage reports whether it sealed and exposesaddress_flooraccordingly;peer_storageguards the POSIX-only import and raises a clear error for the operation.selene: reject Unix-domain manager sockets where they cannot be opened— the Unix arm is gated to Unix platforms and reports the address as unreachable elsewhere.PoolAddresskeeps the variant so manifests and tests that name it stay portable; only the transport that cannot exist is refused.frontend: declare strftime for the Windows C runtime— MSVC's CRT exportsstrftime; the Windows declaration isunsafe externper edition 2024 and the call sits in an explicit unsafe block. Other platforms keeplibc's binding.patches: apply the pinned fixes to the packages the platform installs— the patch stage collects only installed packages, keeps the exact-version requirement for each of them, and skips a bundled patch whose target package is absent. A package the platform cannot install has no files to modify, so its patch has nothing to apply; the version guarantee is unchanged for everything present.assets: keep chat template line endings stable across checkouts— the template assets are declaredtext eol=lf, so every checkout produces the bytes the template digests were computed over.tests: make the helios test profile compile on Windows— theflare-selene-managerdev-dependency and its single consumer (offload::pool::tests) are Unix-scoped, since that crate binds Unix-domain sockets and callsflare_eos::direct_update_file, which exists only ineos/src/unix.rs; theMemAvailableparser is compiled on every platform becausecfg(test)never reaches a dependency; and two cache fixtures use a hyphen instead of a colon, which Windows cannot represent as a regular file name.package: serialise package mutation on Windows—lock_packagetook a lock only undercfg(unix)and did nothing on Windows, so two concurrentemit_derivationscalls both rewrote the package and one derivation was lost. The Windows path now takes a blocking exclusiveLockFileEx.selene: fill the record token from the Windows CSPRNG—shard::nonceread/dev/urandomon every platform, so a two-rank join failed startup on Windows with os error 3 and the peer saw a connection reset. Windows now usesBCryptGenRandomwith the system-preferred RNG.tests: clear the sealed attribute before corrupting a cached payload— the corrupted-bytes warm-hit test relaxed a Unix file mode and could not write the read-only payload on Windows.Validation
omitting released allocation 0xb03800000+692****6016 recorded during preparation.cargo test package::cache(x86_64-pc-windows-msvc,--features runtime-package): 76 passed, 0 failed, 1 ignored (previously 47 passed / 25 failed). Before commit 11 the test profile did not compile on Windows at all, so no helios test could run.cargo testforeos: 5 passed.package::naming: 6 passed on x64 and on the native aarch64 host.runtime_package: 108 passed, 1 failed — see limitations./v1/modelsresponds, eight validation prompts were answered correctly (arithmetic, instruction following, multi-turn, code, JSON, time arithmetic, one-word edge case, ~4k-token needle), and two concurrent requests completed cleanly.heliopausebuilds for Windows ARM64 andflarebuilds in release (2m10s), where neither compiled before.ignition.patches.apply()on Windows ARM64 patchestorch/_inductor'striton_heuristics.pyandstatic_triton_launcher.pyand skips the triton patch; it previously raisedPreparationErrorbefore preparation began.flarerebuilt, the ABI 5 package for Qwen3.8-Flash-Next loads past Load tokenizer (248,320-token vocabulary) and serves: eight of eight validation prompts correct, two concurrent requests completed, median decode 31.4 tok/s over the token window (0.27–0.85 s to first token). Before the change the same package panicked at that phase. An earlier figure of 309.4 tok/s came from timing TTFT on the first content chunk while this checkpoint streams reasoning first, which shrinks the decode window to the tail of generation; the corrected harness counts both channels.Sign-off
All 14 commits carry the DCO sign-off required by CONTRIBUTING.md:
Signed-off-by: R4yharris <harray86@me.com>.Material limitations
authenticated_handles_survive_rename_and_replacementstill fails.eos::windows::direct_read_guarddeliberately reopens a package withFILE_SHARE_READonly to keep it immutable, which prevents renaming the directory the test renames. Narrowing that sharing was tried and reverted (it broke concurrent writers with a sharing violation), so this is left as a maintainer decision on which guarantee wins rather than patched here.sync_file().selene-manageritself is not ported, so the offload pool tests stay Unix-only and Windows has no coverage of those paths.strftimepasses the format to the CRT, so it accepts the CRT's directives rather than POSIX extensions the platform lacks.git add --renormalizeand a re-checkout of the template asset before the line-ending rule takes effect.