Repository navigation
Benchmark packed verification prefixes with a fixed proposal count - #244
Closed
voipmonitor wants to merge 1 commit into
Closed
voipmonitor wants to merge 1 commit into
voipmonitor wants to merge 1 commit into
Conversation
Adaptive verification costs depend on physical target rows while proposal work remains fixed. Rectangular-round benchmarks cannot compare those packed prefixes from one preparation and common prefilled state. Add --verify-prefixes for batch-one benchmarks. Prefixes above one retain the configured logical width and full proposal count; prefix one uses the serving graph without proposals and retains draft-context maintenance. Restore one shared prefilled snapshot for each batch, context, and seed, and report target rows, proposal count, draft-forward size, and acceptance. This enables comparable prefix measurements without repeated model loads. Native serving behavior is unchanged and no inference speedup is claimed. The CPU benchmark suites pass 142 tests covering CLI constraints, selected shapes, row packing, and proposal/target RNG reservations. One existing CUDA snapshot/restore test is skipped; GPU model measurements have not been rerun on this branch. Design-Signed-Off-By: Codex (GPT-6); design-review=cfcf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Measuring adaptive verification cost requires changing the number of target rows while keeping proposal generation fixed. The rectangular-round benchmark cannot compare those packed prefixes in one preparation, and shortening its logical width would also reduce proposal work.
Add
ignition bench --verify-prefixes LISTfor batch one. Prefixes above one retain the configured logical width and full proposal count while selecting the corresponding captured target-row capacity. Prefix one uses the existing width-one serving graph, which omits proposals and retains draft-context maintenance. Cases with the same batch, context, and seed restore one common prefilled snapshot before each replay. Validation checks target and draft metadata and RNG reservations; result JSON records target-prefix size, proposal count, executed draft-forward size, and acceptance separately from GPU latency.This permits packed-prefix comparisons with one model preparation and common input state. It changes the diagnostic benchmark, with no native serving policy or inference-speed improvement claimed. The flag cannot accompany prefill or private speculation studies.
Validation: 142 CPU tests passed, covering CLI constraints, selected graph shapes, row packing, and proposal/target RNG checks. Run
python -m pytest -q tests/bench/test_runner.py tests/cli/test_bench.py tests/preparation/test_benchmark_selection.py tests/recipes/test_kimi_benchmark.py. One existing CUDA graph snapshot/restore test was skipped because CUDA tests were not enabled. Full-model GPU measurements have not been rerun on this branch.