perf(prover): port optimized Akita to the modular prover - #1732
Draft
markosg04 wants to merge 94 commits into
Draft
perf(prover): port optimized Akita to the modular prover#1732markosg04 wants to merge 94 commits into
markosg04 wants to merge 94 commits into
Conversation
github-actions
Bot
added
spec
Contributor
Benchmark comparison (crates) |
This was referenced Aug 3, 2026
…rove seam, byte-diff + e2e harnesses
Reorganize the crate as two parallel prover paths sharing the root
orchestration (config/preprocessing/driver/error): src/dory (the
homomorphic elliptic-curve pipeline, was src/prover.rs + src/stages)
and src/akita (the packed lattice pipeline, port in progress). Like
jolt-verifier, one compiled prover proves exactly one protocol: the
akita feature swaps the wire types to the packed envelope, so exactly
one path module compiles per build.
- akita::prove stub (returns ProverError::Unsupported) pins the packed
seam: backend-first signature over a JoltAkitaBackend registry-to-be,
generic over the scheme — concrete Akita types bind at call sites.
JoltAkitaBackend is a parallel struct rather than cfg-gated
JoltBackend fields: jolt-kernels deliberately has no akita feature
(see jolt-claims' CANONICAL_INSTRUCTION_ADDRESS).
- tests/akita_byte_diff.rs: legacy-vs-new whole-proof ratchets
(muldiv, advice consumer, committed muldiv x {1,2} chunks), all
#[ignore]d until the port lands; component-wise asserts give
per-stage granularity on the packed wire fields.
- tests/akita_e2e.rs: analogs of the legacy packed e2e suite (muldiv,
forced-K256, advice, full-advice, committed program), #[ignore]d;
un-ignoring them is the port's acceptance gate.
- tests/dory_byte_diff.rs (was byte_diff.rs): modules re-gated
all(prover-fixtures, not(akita)) — the two harnesses are mutually
exclusive by feature.
- jolt-kernels: the bytecode read-raf reference kernel takes the
jolt-claims NUM_BYTECODE_VAL_STAGES seam (5 base / 6 akita) and
guards its base-five fold at runtime; byte-neutral in base mode
(the dory_byte_diff muldiv ratchet passes).
- Cargo: akita = [jolt-verifier/akita, jolt-claims/akita,
jolt-field/akita, jolt-prover-legacy/akita] — legacy must flip
together with the verifier; the verifier's own akita edge to it is a
dev-dep edge that does not propagate.
- CI: clippy lanes for -p jolt-prover --features
akita[,prover-fixtures] so the ignored suites keep compiling.
Verified: clippy -D warnings on {default, prover-fixtures, akita,
akita+prover-fixtures}; workspace clippy under host and host,zk;
nextest default and akita lanes; dory_byte_diff muldiv end-to-end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cross-evolution integration that belongs to no single branch commit:
jolt-utils extraction re-points (jolt_poly::{math,thread} moved), the
stage-flamegraph port into dory/prover.rs, profile/zk_e2e adaptation to
the borrowed TraceBackend witness API, session-witness removal from
ProofSession (superseded by owned_rows), and byte-diff harness
reconciliation.
0xAndoroid
force-pushed
the
perf/akita-prover-opt
branch
from
August 6, 2026 19:07
c10b269 to
cbf59c9
Compare
The Akita port removed the last jolt_utils usage from jolt-witness (cargo-machete failure). The parallel feature's jolt-utils/parallel forward goes with it: jolt-kernels, the only jolt-witness/parallel consumer, already enables jolt-utils/parallel directly.
0xAndoroid
force-pushed
the
perf/akita-prover-opt
branch
from
August 7, 2026 03:16
9201dc7 to
816d3af
Compare
# Conflicts: # crates/jolt-verifier/src/stages/stage8/reconstruction.rs
Regenerated with JOLT_FS_BLESS=1 cargo nextest run -p jolt-verifier --test fs_obligations --features fs-audit after merging main (which brought #1702's gate to this branch). Unlike a mechanical re-spell, these diffs are this PR's intentional protocol-surface changes; for the security review they group as: - Packed-opening redesign: the jolt-openings packing.rs batch absorbs and its three challenge draws are replaced by the prefix.rs PrefixPackedLayout::reduce_claims schedule (layout digest + logical num_vars/slot capacity absorbs, one challenge_vector draw). - Akita domain separation: append_verifier_setup now absorbs the b"akita/fp128" label plus a runtime ring_dimension instead of the baked d64 label/AKITA_D constant; AkitaCommitment additionally absorbs ring_dimension before backend_coeff_len and the bytes. - Committed-program plurals: absorb_commitments defers to new absorb_packed_commitments/absorb_packed_program_commitments helpers over program_one_hot_commitments (source schema: program_one_hot_commitment(s)/program_one_hot_setup(s) renames).
Upstream review (akita#344, closed) rejected baking a payload-slack policy into catalog identity. Remove the preset slack overrides, pin akita at upstream main, and regenerate the K16/K256 catalogs under the default min-payload objective. Geometry pin tests now assert D64 rank 7 (P=2^20) and D128 rank 4 (P=2^18, 11 GiB setup envelope).
# Conflicts: # crates/jolt-claims/src/protocols/jolt/lattice/packing.rs # crates/jolt-openings/Cargo.toml # crates/jolt-openings/src/packing.rs # crates/jolt-prover-legacy/src/zkvm/packed.rs
jolt-utils now owns the Math trait; the generated catalog headers are rustfmt-formatted (the emitter's fixed import list is not fmt-stable), gen_jolt_schedules formats its output, and the drift oracle compares schedule data with the import boilerplate stripped.
# Conflicts: # Cargo.lock # Cargo.toml # crates/jolt-akita/Cargo.toml # crates/jolt-akita/src/adapters.rs # crates/jolt-akita/src/configs.rs # crates/jolt-akita/src/native_batching.rs # crates/jolt-akita/src/schedules/jolt_fp128_d64_onehot_k16.rs # crates/jolt-akita/src/schedules/jolt_fp128_d64_onehot_k256.rs # crates/jolt-akita/src/scheme.rs # crates/jolt-claims/src/protocols/jolt/lattice/packing.rs # crates/jolt-prover-legacy/src/zkvm/packed.rs # crates/jolt-verifier/src/stages/stage8/packed.rs # crates/jolt-verifier/src/verifier.rs
The prefix-packed statement adds a fifth constructor argument; the Dory-shape fixture carries no packed RAM hamming weight.
# Conflicts: # crates/jolt-prover-legacy/src/zkvm/packed.rs
Pins akita at the reconciled prover stack (flattened setup, compressed payloads, catalog-row schedule selection). The adapter moves to self-describing committed groups — verify reconstructs each group's profile from the trusted catalog row while proofs carry payload coefficients only — and batched prove/verify thread the public row-digest selection. The verifier accepts only cataloged rows now, so every provable shape is an approved row: the D64 one-hot grids extend down to the adapter test shapes, and D128 one-hot and dense advice/program families gain their own generated catalogs (the dense fp128 D64 floor is nv=14; smaller objects pad up). Geometry pins re-derived under upstream's evolved sizing: D64 K256 selects rank 6 at P=2^20 and D128 rank 3 at P=2^19 natively — most of the retired payload-slack policy's geometry returns as the min-payload choice — and the D128 setup capacity drops 11 GiB -> 4.125 GiB under prefix-exact accounting.
Pin Jolt to the current LayerZero Akita commit and migrate the fixed-dimension configs, generated catalog identity, CPU backend, grouped root commitment, wide-ring, and sparse challenge APIs. Preserve Jolt's shared matrix cache release boundaries with Akita's explicit root-fold policy while leaving compression NTT state resident. Propagate cleanup failures through the prover error path.
Merge main at 84bf9f2. Keep the PR's consolidated registers test support and renamed Dory harness, then port main's advice tape contract into those current locations. Preserve captured advice tape data through padded and field inline trace reconstruction instead of discarding it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The modular prover lacks an Akita path and still materializes several trace-sized tables that the optimized backend can derive or stream. Stacked on #1731, this ports the optimized kernel families, adds modular Akita proving and reconstruction, streams packed commitments, compacts witness storage, and releases large tables after their final stage.
Local
sha2-chainacceptance runs at the default minimum-payload schedules reached 51.67 s / 26.22 GiB at 2^26 and 205.45 s / 76.41 GiB at 2^28; the 2^28 run completed without swapouts. (The payload-slack planner policy that previously improved these to 49.78 s and 173.48 s was dropped from the stack per upstream planner review — LayerZero-Labs/akita#344 is closed; the measured trade is recorded in the acceptance spec for a future planner-native mechanism.)The akita dependency has since been reconciled with akita
main(flattened setup, compressed payloads, catalog-row schedule selection), and the jolt side ported: committed groups are self-describing (verify takes each group's profile from the trusted catalog row; proofs carry payload coefficients only), and since the verifier now accepts only cataloged rows, every provable shape is an approved row — D128 one-hot and dense advice/program objects gain their own generated catalogs. Under upstream's evolved sizing the min-payload planner natively selects D64 rank 6 (P=2^20) and D128 rank 3 (P=2^19) — most of the geometry the retired slack policy bought — and the D128 setup capacity drops from 11 GiB to 4.125 GiB with prefix-exact accounting. The byte-diff suites confirm the modular prover still matches the legacy prover on the reconciled substrate. The 2^26/2^28 numbers above predate the reconciliation; the acceptance ladder needs a re-run before merge.Changes
Testing
cargo clippyandcargo fmtpassAkita e2e and byte-parity suites, 122 optimized-kernel tests, RAM fallback tests, and both required clippy feature modes pass.
Security Considerations
The protocol and verifier changes are reviewed in #1731. Differential kernel tests compare optimized outputs with the reference backend, and the e2e suites verify the resulting proofs rather than only comparing intermediate values.
Breaking Changes
Inherits the Akita proof/setup changes from #1731. The modular prover's internal backend and stage APIs change to support Akita and streamed witness sources.