You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f7c468c
Browse filesBrowse the repository at this point in the historyBrowse files
Fix TP16 host prefix records and retain deep DRAM prefixes (#264)
* Fix TP16 host prefix records and retain deep DRAM prefixes
Version 1 and 2 record headers cannot describe the Kimi TP16 geometry: sixteen ranks with 768 planes require 6513 bytes, exceeding the fixed 4096-byte header block and rejecting every host-cache publication.
Add record format version 3 with u32 plane sizes, reducing that header to 3441 bytes without changing segment offsets. Preserve version 1 and 2 decoding and byte-identical rewrites, keep version metadata out of semantic header equality, and reject plane sizes that do not fit the new encoding.
Keep redundant interior rows while the DRAM pool has more than two row slots of headroom. Resume eager row thinning automatically under real memory pressure so deep-prefix restore does not trade unbounded DRAM growth for retention. Update retention tests to cover direct deep-endpoint restores and pressure behavior.
Log host-store submission and completion errors instead of silently discarding their causes.
Validation: all 393 Selene unit tests pass. Native TP16 store/drain and DRAM restore tests show zero I/O errors and a complete 153295-token prompt reused as 132593 GPU tokens plus 20702 host-restored tokens with zero new prefill.
Those service tests used the db29 C1 diagnostic package. A separate C4 long-request CUDA_ERROR_ILLEGAL_ADDRESS remains under investigation and is not covered by this commit. Previous-core serving beyond signed package inspection was not separately revalidated.
Design-Signed-Off-By: Kimi (Kimi); design-review=cfcf
Co-Authored-By: Kimi <noreply@anthropic.com>
* Preserve host record compatibility and reclaim rows under pressure
Reserved transfer intake made the row-thinning threshold unreachable during
ordinary retention pressure. Reclaim eligible redundant rows using free
warm slots, and do so before incoming allocations evict reusable endpoints.
Remove queued thinning keys with removed or compacted records so bookkeeping
remains bounded by the resident index. Pinned rows remain protected.
Record validation used the v3 size limit even for decoded legacy headers,
and cross-rank agreement hashed wire versions despite semantic equality.
Validate the stored encoding, retain lossless legacy writes for wide planes,
select valid legacy replication versions, and hash a canonical encoding for
rank agreement. Existing record bytes remain stable when rewritten without
semantic changes. Update the cache format and retention documentation.
These changes preserve usable endpoints under pressure and keep equivalent
records from being rejected across format versions. Metadata-only tests cover
4 GiB planes, byte-exact rewrites, replication flags and mixed-format agreement.
Pressure tests cover intake reserves, multi-slot rows, larger incoming records,
pins and repeated chain removal. All 401 Selene tests and the CLI cargo check
pass. No GPU or end-to-end serving test was run during this review.
Design-Signed-Off-By: Codex GPT-6; design-review=cfcf
Signed-off-by: Martin Vit <martin@voipmonitor.org>
---------
Signed-off-by: Martin Vit <martin@voipmonitor.org>
Co-authored-by: Kimi <noreply@anthropic.com>
0 commit comments