Repository navigation
fix(helios): bound history proposals by logical lane length - #289
shadownetX wants to merge 1 commit into
Conversation
History drafting uses the captured graph width even when the scheduler limits a lane to a shorter output or context tail. A rectangular verifier can then report fewer offered drafts than the copy provenance mask represents, violating the bounded completion contract and terminating serving. Limit the history lookup in input packing to the scheduled lane length minus the pending token rather than the physical graph width. Extend the shared-history upload regression to cover a width-eight graph with a two-token lane and a longer available history match. This preserves the copy proposal bound for host-selected rectangular lanes without changing graph preparation or weakening completion validation. The targeted CPU test fails without the fix and passes with it. A release ARM64 build based on v0.1.0 served the existing Qwen3.8 Flash Next core on a single GB10 with native MTP, adaptive verification, mixed prefill and history drafts enabled. The previously failing two-request scenario and the complete cold/warm benchmark campaign passed without request errors. The broader native-serving CI still has failures also observed on the unmodified base; the native runtime step was skipped. Additional device-side reductions of packed lane ceilings and TP configurations are not qualified. Design-Signed-Off-By: ChatGPT (GPT-6); design-review=ed22 Signed-off-by: vonP <thomas@picollin.fr> Signed-off-by: ChatGPT <support@openai.com> Co-authored-by: ChatGPT <support@openai.com>
original-el8
added
area:scheduling
|
Categorized as The source change bounds history proposals by the scheduler-selected lane length. The regression deliberately keeps graph capacity at eight while shortening the logical lane to two, then expects only one history proposal. That directly exercises the mismatch behind the reported copy-provenance bound; completion validation is unchanged. The author reports a failing-before/passing-after CPU regression and successful GB10 TP1 native serving of the previously failing scenario and full campaign, reusing the prepared core. Serving on current upstream main, additional packed-verification narrowing and other TP configurations remain untested. This triage checked source and mergeability, not a new GPU run. Head |
Fixes #287.
History drafting previously used the captured graph width even when the scheduler limited a request to a shorter logical lane. This could produce a copy provenance mask exceeding the offered-draft bound and terminate serving.
The fix limits history proposals to
lane.tokens.saturating_sub(1)and extends the shared-history worker regression to cover shortened speculative lanes. Completion validation remains unchanged.Validation
Validation covers native MTP with adaptive verification and mixed prefill on Qwen3.8 Flash Next. Additional packed verification narrowing and TP configurations remain untested. Serving on current upstream main has not yet been validated.