refactor: adaptive ring dimension - #356
Conversation
Documentation blast radius (advisory)These regions may need doc/spec/book updates based on changed paths. Changed files in this PR: 127
|
github-actions
Bot
added
spec
CI test timing
Run summary
Wall time spans 2 parallel nextest slice shards. Slowest tests
Regressions vs main
New slow testsNo new tests ≥30s vs main baseline. |
Benchmark Report
Terminal response component breakdown
The Detailed schedule and proof-size breakdowns by fold level are available in the uploaded |
# Conflicts: # .github/workflows/profile-bench.yml # book/src/usage/profiling.md # crates/akita-config/Cargo.toml # crates/akita-config/src/lib.rs # crates/akita-config/src/precommitted_commitment.rs # crates/akita-config/src/recursive_commitment.rs # crates/akita-config/tests/generated_tables.rs # crates/akita-config/tests/runtime_fallback.rs # crates/akita-config/tests/schedule_catalog_miswire.rs # crates/akita-pcs/Cargo.toml # crates/akita-pcs/examples/profile/modes.rs # crates/akita-pcs/tests/support/mod.rs # crates/akita-planner/examples/mixed_dimension_search.rs # crates/akita-planner/src/emit/mod.rs # crates/akita-planner/src/generated_families.rs # crates/akita-planner/src/planner.rs # crates/akita-planner/src/schedule_params.rs # crates/akita-planner/src/test/schedule_params.rs # crates/akita-schedules/Cargo.toml # crates/akita-schedules/src/catalog_identity.rs # crates/akita-schedules/src/generated/fp128_d128_dense.rs # crates/akita-schedules/src/generated/fp128_d128_onehot.rs # crates/akita-schedules/src/generated/fp128_d256_onehot.rs # crates/akita-schedules/src/generated/fp128_d64_dense_multi_chunk.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_multi_chunk.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_multi_chunk_w2r2.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_multi_chunk_w4r2.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_recursive.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_recursive_multi_chunk_w8r2.rs # crates/akita-schedules/src/generated/fp128_d64_onehot_tensor.rs # crates/akita-schedules/src/generated/fp128_dense.rs # crates/akita-schedules/src/generated/fp128_mixed_dim_onehot.rs # crates/akita-schedules/src/generated/fp128_onehot.rs # crates/akita-schedules/src/generated/fp32_d128_onehot.rs # crates/akita-schedules/src/generated/fp32_d256_onehot.rs # crates/akita-schedules/src/generated/fp64_d128_dense.rs # crates/akita-schedules/src/generated/fp64_d128_onehot.rs # crates/akita-schedules/src/generated/fp64_d256_onehot.rs # crates/akita-schedules/src/generated/mod.rs # scripts/profile_bench_report.py
|
Omid, thank you for the work here. This is a nice direction: making the common fp128 direct one-hot and dense paths choose ring dimensions from the actual workload is much better than encoding a single dimension everywhere. I reviewed head What this PR doesSuggested title:
The PR makes adaptive ring-dimension schedules the default for the direct fp128 one-hot and dense presets. The flat public matrix remains dimension-free; the selected schedule chooses the protocol dimensions. For the adaptive search, A may use D64, D128, or D256, while B/D use the configured supported choices and the suffix settles to D64. The selected schedules are emitted into generated catalogs and consumed by the runtime configuration path. Recursive and multi-chunk adaptive search remain follow-up work. flowchart LR
S["Dimension-free flat public matrix"] --> W["Workload: shape, density, grouping"]
W --> C["Enumerate legal A/B/D schedules"]
C --> O["Apply security and sizing constraints"]
O --> R["Rank candidates by the normative objective"]
R --> G["Generated fp128 catalog"]
G --> P["Prover and verifier use the selected schedule"]
Representative generated behavior:
The change also adds useful pieces that look worth keeping: FindingsBlocker — the adaptive policy reintroduces the removed setup-generation/carrier dimensionThe flat-public-matrix cutover removed setup generation dimension entirely: setup capacity is measured in base-field elements, and the schedule is the sole owner of A/B/D dimensions. The mixed-dimension spec says explicitly that However, An adaptive policy should validate each proposed scheduled dimension against the supported ring/security/challenge dispatch directly. It should not require all dimensions to divide a global Blocker — the implemented objective disagrees with the active normative objectiveThe active mixed-dimension spec defines the comparison key as: The PR instead introduces That changes which schedule is canonical. Either restore the setup-first objective in code and generated catalogs, or update the normative spec and justify the protocol-wide policy change before making this the default. Major — candidate deduplication can discard the winner under the PR's own rank-first objective
The first pass keeps X because its setup is smaller, so the later rank-first comparison never sees Y. This contradicts Major — suffix frontier pruning is not rank-sound for the accepted search-depth rangeThe policy accepts The exhaustive-oracle coverage that would establish this invariant was removed for the adaptive case; the remaining unpruned comparison covers only uniform D64. Please either make the frontier key/order preserve the full future-relevant rank vector, restrict the supported adaptive depth to the proven case, or restore an exhaustive comparison over representative adaptive domains and depths. Major — planner and schedules crates now contain parallel canonical implementationsThe PR duplicates This is exactly the kind of split-brain sizing/schedule logic that becomes hard to audit: generation can certify one implementation while runtime materializes or prices another. Please put the shared candidate model and pricing/materialization primitives behind one canonical implementation and have both paths call it directly. Moderate —
|
|
I agreed with the findings and addressed them in Point-by-point:
Additional notes:
Validation after these changes:
|
Remove the obsolete setup generation dimension from live requirements. Document the current local B/D rank choice as a P0 limitation and keep full Cartesian optimization as the target.
PR SummaryMedium Risk Overview Breaking surface area spans Cargo features ( Planner/catalog emission binds Reviewed by Cursor Bugbot for commit 8e243cb. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
Omid, thank you again for the work on this. The PR is a useful improvement, and I am comfortable treating the remaining B/D search issue as explicit follow-up work so this stack can keep moving. Follow-up: apply the canonical objective to the full B/D domainProduction currently enumerates A dimensions, then An independent full-domain probe found this concrete case for fp128 one-hot at
The explicit tuple wins by 112 proof bytes at the same setup cost. The current checked-in unpruned test varies A while holding B and D at D64, so it cannot detect this case. I pushed The follow-up should:
This note is intended to preserve the measured limitation and the next correction, not to hold up the current stacked PR. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 8e243cb. Configure here.
* feat(schedules): support adaptive multi-chunk rings * feat(schedules): support adaptive recursive rings * fix(schedules): include adaptive recursive precommits * chore(schedules): regenerate after planner rebase * fix(planner): preserve fixed-D64 tie regression
…ense Remove fixed-D64 fp128 preset types and fp128_d64 schedule families now superseded by adaptive Dense/OneHot catalogs, including recursive setup offloading and Jolt recursion pinning. Co-authored-by: Cursor <cursoragent@cursor.com>
# Conflicts: # crates/akita-planner/src/schedule_params.rs # crates/akita-schedules/src/generated/fp128_d128_dense.rs # crates/akita-schedules/src/generated/fp128_d128_onehot.rs # crates/akita-schedules/src/generated/fp128_d256_onehot.rs # crates/akita-schedules/src/generated/fp128_dense.rs # crates/akita-schedules/src/generated/fp128_dense_multi_chunk.rs # crates/akita-schedules/src/generated/fp128_mixed_dim_onehot.rs # crates/akita-schedules/src/generated/fp128_onehot.rs # crates/akita-schedules/src/generated/fp128_onehot_multi_chunk.rs # crates/akita-schedules/src/generated/fp128_onehot_multi_chunk_precommitted.rs # crates/akita-schedules/src/generated/fp128_onehot_multi_chunk_w2r2.rs # crates/akita-schedules/src/generated/fp128_onehot_multi_chunk_w4r2.rs # crates/akita-schedules/src/generated/fp128_onehot_multi_chunk_w4r2_precommitted.rs # crates/akita-schedules/src/generated/fp128_onehot_recursive.rs # crates/akita-schedules/src/generated/fp128_onehot_recursive_multi_chunk_w8r2.rs # crates/akita-schedules/src/generated/fp128_onehot_recursive_multi_chunk_w8r2_precommitted.rs
3 participants
Adaptive ring dimensions for default fp128 schedules
Summary
This PR makes adaptive ring-dimension schedules the defaults for direct, scalar fp128 one-hot and dense commitments.
The public setup matrix remains a dimension-free flat field-element stream. Setup capacity is measured directly in base-field elements; there is no setup-generation or carrier ring dimension. The selected schedule is the sole owner of the per-level A/B/D dimensions.
Adaptive planning happens offline during catalog generation. Proving and verification resolve and validate the generated schedule row and never invoke the planner at runtime.
What changed
Declarative ring-dimension policy
CommitmentConfigdescribes schedule behavior withRingDimensionScheduleMode:uniform_ring_dimensionis only the candidate used by uniform presets. It is not an adaptive setup carrier, and adaptive dimensions do not need to divide it.Each adaptive dimension is instead validated directly against its protocol role:
Bounded adaptive search
The default
fp128::OneHotandfp128::Densepolicies use:Dimensions are component-wise non-increasing across the adaptive boundary. The currently audited implementation accepts exactly two adaptive levels; other adaptive depths are rejected by policy validation.
Complete schedules use the normative deterministic comparison key:
There is no rank-first objective or preliminary dimension-path deduplication. Secure matrix ranks are feasibility and sizing inputs; all complete candidates are compared once using the canonical objective.
One canonical schedule implementation
The candidate representation and shared pricing/materialization operations now live in
akita-schedulesand are called directly byakita-plannerand generated-row replay. This includes:This removes parallel planner/runtime implementations and keeps generation, replay, setup accounting, and proof pricing on the same primitives.
Generated catalog identity and runtime replay
Generated catalog identity binds the complete schedule policy, including:
Validation checks that emitted schedules use admitted dimensions, obey the adaptive-to-D64 cutover, match their exact proof/setup estimates, and expand to the same canonical descriptor produced by the planner. Missing runtime rows reject rather than falling back to planner execution.
fp128 preset and catalog cleanup
fp128::OneHotis the canonical adaptive direct one-hot preset.fp128::Denseis the canonical adaptive direct dense preset.D64OneHotandD64Densepresets remain available as baselines.D256OneHotremains a tableless experimental policy marker.fp128_mixed_dim_onehotnaming were removed.profile-onehot-fp128-d64toprofile-onehot-fp128.best_onehot_schedulecompatibility layer and its fabricated estimate breakdown were removed; callers use the standard generated-catalog path directly.Scope and limitations
This PR implements adaptive planning for direct, scalar fp128 one-hot and dense schedules.
The following remain outside this cut:
Grouped paths can preserve already-committed profiles, but they do not perform planner-native mixed-D multi-group optimization. Recursive and multi-chunk fp128 configurations continue to use their existing uniform-D64 catalogs.
Compatibility
This is an intentional breaking configuration and generated-catalog change. Users of removed fp128 generated families or the old mixed-dimension naming should migrate to:
Use the corresponding
D64*preset when an explicitly uniform D64 policy is required.Validation
Coverage includes:
layout.d_a()The full CI-equivalent test graph passes locally: 1,254 tests across 50 binaries, along with all required Clippy configurations and repository preflight checks.