perf(sis): probe ADPS16 L2 rank frontiers

[quangvdao]

Caution

This is an unsound diagnostic. The prover sends an observed squared L2 norm and the verifier binds the payload to the transcript, but no sumcheck proves the norm relation. The diagnostic also assumes a negacyclic-convolution operator-norm rejection cap Γ = 17 that the sampler does not enforce. This PR makes no production security claim and must not merge as written.

Summary

This draft measures whether a late-fold L2 certificate can reduce Ajtai A ranks and proof size on top of PR #343.

It now:

• uses the 128-bit quantum ADPS16 Euclidean SIS model, not the retired BDGL16 path;
• prices the complete scalar collision vector without multiplying its norm by width a second time;
• uses the challenge's negacyclic-convolution operator norm in the collision formula;
• retains one recursive planner candidate per secure A rank, plus the ordinary L-infinity fallback;
• sends and accounts for the unchecked L2 diagnostic payload at late single-chunk folds;
• measures fp128, fp64, and fp32 through the normal generated-schedule and profile paths.

The corrected small-field result is narrower than the earlier experiment. At Γ = 17, fp32 D128 saves 1,989 sampled bytes versus PR #343, while fp64 D128 grows by 114 sampled bytes. Neither profile reduces its fold count. The previously reported five-block tails depended on substituting coefficient ||c||₂ for convolution operator norm and are not a defensible security result.

Diff metadata

• Base branch: quang/compressed-commitments-pr341
• Current base tip: 25cb450c19ea4ea70b548631f7f6f12c16f67630
• Merge base: b7b9290061167eb25c04e95882719de3a34a18b4
• Head: be2fbe5c4f0a21707020ef1394d8232d500b3c8e
• Commits: 4
• Files changed: 55
• Diff: 2,317 insertions and 642 deletions
• Status: Draft
• Stack base: PR feat(compression)!: add compressed prefix and raw suffix payloads #343

Motivation

Collision security is priced in Euclidean norm by ADPS16. The existing folding path certifies an L-infinity envelope, which can be much more conservative for a late folded witness whose energy is spread over many coordinates.

The diagnostic asks a concrete question: if a future protocol proves a whole-witness L2 cap only at late folds, which A ranks and suffix schedules would the planner select, and what proof bytes would that save?

Two planner details matter.

1. A locally smallest rank is not necessarily the globally cheapest suffix. Rank changes feed back into decomposition basis, block geometry, and the next witness length.
2. Ring challenge multiplication is negacyclic convolution. Its L2 security price is controlled by convolution operator norm, not by coefficient L2 norm.

Change-surface overview

Area	Before	This draft
Euclidean SIS model	Older or width-double-counted paths	Complete scalar collision norm under quantum ADPS16
A collision formula	L-infinity challenge mass only	Optional unchecked L2 price using Γ² and embedding operator norm
Recursive planner	One locally selected matrix	One best split per distinct A rank, plus L-infinity fallback
Prover	No whole-witness L2 payload	Checked u128 accumulation and diagnostic serialization
Verifier	No L2 payload	Shape validation and transcript binding, but no relation check
Proof-size model	No diagnostic bytes	Exact payload bytes and one extra Stage 1 coefficient per final leaf round
Experiments	Fixed first grind nonce	AKITA_L2_DIAGNOSTIC_GRIND_START varies every fold level

Exact security formula

Let

S = ||z||₂²
Γ = op_norm(c)
E = ||ψ||op²

where Γ is the negacyclic-convolution operator norm of the ring challenge and E is the squared L2 operator norm of the fixed trace-subfield embedding.

The two-response extraction factors contribute 8², so the complete scalar A-collision bound is

C_A = ||collision||₂² <= 64 · Γ² · E · S.

For current profiles:

• q128 uses the base-field path, so E = 1;
• q64 and q32 use paired-lane embeddings, so E = 2.

At the diagnostic Γ = 17 and S_cap = 2³²:

q128 multiplier       = 64 · 17²     = 18,496
small-field multiplier = 64 · 17² · 2 = 36,992

q128 exact collision       = 79,439,715,106,816  -> bucket 2^47
small-field exact collision = 158,879,430,213,632 -> bucket 2^48

The SIS instance uses

n = rank · D
m = width · D
length_bound = sqrt(C_A).

Width is already present in m. It must not multiply C_A again.

The generated Euclidean tables use the 128-bit quantum ADPS16 cost model. This PR removes the older BDGL16 route.

Why coefficient L2 is not the operator bound

For the D128 production shell, every challenge has 31 coefficients of magnitude one, so

||c||₂² = 31.

That number controls average random-sign energy:

E_c[||c · w||₂² | w] = 31 · ||w||₂².

It does not bound worst-case ring multiplication. Negacyclic multiplication by c has operator norm

Γ² = max_j |Σ_k c_k ζ_j^k|²,

over the negacyclic roots ζ_j. Security needs this maximum. Rejection sampling may enforce a smaller Γ, but accepted challenge entropy must still be proved separately.

ADPS16 table and bucket semantics

The checked-in L2 tables map

(modulus profile, ring dimension, rounded collision_l2_sq, width)
    -> minimum secure module rank.

The maximum 2^84 value in bucket rounding is table coverage, not a witness cap. The witness diagnostic cap is the separate S_cap = 2^32.

To expose globally useful rank paths without adding a second serialized rank field, the planner conservatively widens the collision bucket through supported powers of two and keeps one candidate whenever the secure rank changes. Every retained bucket upper-bounds the same exact collision and is rechecked by the ADPS16 table during schedule expansion.

Candidates with the same A rank are equivalent for downstream geometry, so the frontier is deduplicated by rank. Deduplicating by (rank, bucket) caused many equivalent suffix states and excessive catalog-generation memory.

Planner root cause and fix

The original diagnostic constructed the ordinary L-infinity matrix and overwrote it when a smaller L2 matrix existed. That removed higher-rank paths before suffix optimization.

The fixed search:

1. constructs each distinct secure L2 A rank;
2. appends the ordinary L-infinity A matrix independently;
3. evaluates every block split for each rank;
4. keeps the best split per A rank;
5. passes the complete rank frontier into the memoized suffix DP.

This preserves the globally useful alternative while keeping one canonical matrix-construction path. Test-only unpruned oracle fixtures were reduced from arity 16 to arity 12 because they intentionally materialize every complete suffix and otherwise become exponential once legitimate rank branches exist. The arity-28 small-field regression covers the production-size frontier separately.

Prover and verifier diagnostic path

At a marked late fold, the prover:

• accumulates the undecomposed folded-witness squared norm with checked u128 arithmetic;
• serializes the norm and diagnostic coefficient payload;
• absorbs the payload into the transcript;
• charges the Stage 1 placeholder coefficient that a future norm sumcheck would add.

The verifier:

• validates the diagnostic payload shape;
• absorbs the same values into the transcript;
• does not prove S = ||z||₂²;
• does not check S <= S_cap;
• does not enforce op_norm(c) <= 17.

A marked L2 candidate cannot become a direct terminal response because the terminal proof does not carry this diagnostic payload.

Corrected fp64 and fp32 results

These runs use D128 one-hot, num_vars = 28, the regenerated catalog, Γ = 17, and the ordinary non-ZK terminal response.

Profile	PR #343	This draft	Change	Fold bytes	Tail bytes	Levels
fp64 D128	83,602 B	83,716 B	114 B more (+0.14%)	26,584 B	57,132 B	6
fp32 D128	77,831 B	75,842 B	1,989 B less (-2.56%)	26,844 B	48,998 B	7

Terminal Golomb payload varies slightly with the sampled witness. Fold-byte changes are structural.

fp64 schedule

root A rank:             5
recursive A ranks:       5, 5, 5, 4
recursive log bases:     4, 6, 6, 6
L2-priced folds:         last recursive fold only
L2 collision bucket:     2^48
terminal A rank:         4
terminal geometry:       864 live rings, 128 positions, 7 blocks

The diagnostic does not beat PR #343 for fp64, so a production planner should leave it disabled for this profile.

fp32 schedule

root A rank:             9
recursive A ranks:       10, 9, 7, 7, 7
recursive log bases:     3, 4, 4, 6, 6
L2-priced folds:         last three recursive folds
L2 collision bucket:     2^48
terminal A rank:         8
terminal geometry:       760 live rings, 128 positions, 6 blocks

The rank-7 late path saves proof bytes, but it does not reduce the number of folds.

Why the five-block tail disappeared

The invalid coefficient-L2 experiment rounded the small-field collision to 2^44. At that bucket the planner found:

• fp64 late ranks 3,3,3 and a five-block terminal;
• fp32 late ranks 6,6,6 and a five-block terminal;
• sampled totals of 75,929 B and 73,136 B.

Those numbers are useful only as a diagnostic upper bound on the optimization opportunity. They are not a security result.

With S_cap = 2^32 and E = 2, reaching bucket 2^44 requires

128 · Γ² · 2^32 <= 2^44
Γ² <= 32.

Under the current integer threshold representation, that means Γ <= 5. Parseval gives

Γ >= ||c||₂ = sqrt(31) ~= 5.57,

so an integer cap of five accepts no challenge in this shell. Experiments at Γ = 17, 12, and 8 all produced the same seven-block fp64 and six-block fp32 tails. The fold-count reduction therefore does not survive the correct operator-norm model.

Norm sweep

I varied AKITA_L2_DIAGNOSTIC_GRIND_START over 32 values. This changes every level's fold challenge while holding the polynomial and initial seed fixed.

Profile/fold	Maximum observed S	Share of 2^32 cap	Share of norm radius
fp64 L4	1,986,395,942	46.25%	68.01%
fp32 L3	412,518,025	9.60%	30.99%
fp32 L4	232,699,708	5.42%	23.28%
fp32 L5	1,552,260,821	36.14%	60.12%

All 128 marked-fold observations were below 2^32. This supports honest-prover liveness for the sampled workloads. It does not replace a concentration argument or a verifier-enforced cap.

Small-field embedding and the terminal

The factor E = 2 appears because the fixed q64/q32 trace-subfield embedding has squared L2 operator norm two. It is part of recursive A-collision pricing and cannot be dropped merely because the outer field is small.

The current terminal is still priced and checked through the existing L-infinity path. The clear terminal witness is sent directly and is not decomposed into a next-level z_hat, so this PR does not apply an L2 embedding factor or an L2 rank cutover there. A future clear-witness terminal analysis should derive its own kernel/operator bound from the verifier's direct checks rather than automatically copying the recursive embedding factor.

Sound protocol path

No zero-knowledge machinery and no four-squares proof is needed for the non-ZK rollout described here.

1. Stage 1 proves the claimed complete ||z||₂² with a batched sumcheck.
2. Stage 2 links z to the existing digit-decomposed virtual witness z_hat.
3. The verifier checks the proven norm against the cap attached to the selected A matrix.
4. The existing L-infinity range certificate remains in force.
5. The challenge sampler enforces op_norm(c) <= Γ, and an exact counting or rigorous lower-bound argument proves at least 128 bits of accepted challenge support.

For a small base field, the full integer norm may wrap. If

z = Σ_a B^a · z_hat_a,

the verifier can instead certify the limb Gram terms

G[a,b] = <z_hat_a, z_hat_b>
||z||₂² = Σ_(a,b) B^(a+b) · G[a,b].

Symmetry keeps only a <= b, with a factor of two off diagonal. Reconstruction still needs a large enough integer range or extension field to exclude modular wraparound.

Security and verifier safety

This draft deliberately stops before sound verification.

• The verifier validates shape and transcript binding.
• Norm accumulation uses checked arithmetic.
• ADPS16 sizing and schedule expansion use the same collision bucket.
• The L-infinity certificate remains independent.
• The missing norm sumcheck and missing operator-norm rejection are explicit.
• No claim in this PR treats the diagnostic rank as production-secure.

If both L-infinity and L2 checks exist in a future protocol, accepted collisions lie in the intersection of the two sets. The planner may use whichever independently sound analysis yields the smaller secure rank. The estimates are not multiplied or blended.

Breaking changes and preserved behavior

This is a diagnostic protocol and generated-identity change. Marked L2 schedules add transcript and wire data and therefore are not compatible with proofs generated before this branch.

Unmarked schedules retain the existing layout and do not gain a descriptor byte. L-infinity-only pricing, multi-chunk early folds, tensor challenges, recursive setup prefixes, and direct terminal verification remain supported.

Commit map

• e6726f9d9 — add the unchecked ADPS16 L2 planning and wire probe;
• 49ad5b24b — account for the complete diagnostic proof payload;
• 513bba7e7 — preserve recursive A-rank frontiers in suffix planning;
• be2fbe5c4 — replace coefficient L2 with convolution operator-norm pricing.

Validation at be2fbe5c4

Local validation completed:

• full schedule regeneration;
• akita-types: 380 passed, 1 ignored;
• akita-planner --features catalog-gen: 35 passed;
• targeted release Clippy for akita-types and akita-planner;
• all three repository CI Clippy feature graphs;
• Rustfmt and Taplo;
• Rust line-cap checks;
• 34 Python script tests;
• verifier, prover, config, planner, and setup dependency guards;
• Cargo Machete and typos;
• fp64/fp32 prove-and-verify profiles;
• 32 grind-start samples for both small-field profiles.

GitHub checks restarted for this exact head and are still pending. Earlier-head green checks are not counted as validation of be2fbe5c4.

Remaining work

A production rollout still needs:

1. the Stage 1 norm sumcheck;
2. the Stage 2 link to digit-decomposed z_hat;
3. the small-field limb-Gram/no-wrap path;
4. verifier-enforced per-level norm caps;
5. exact operator-norm rejection in the challenge sampler;
6. a rigorous accepted-support certificate at the selected Γ;
7. per-level energy classes if the planner moves from one global cap to tighter state-dependent caps;
8. a separate clear-terminal security derivation;
9. broader tail measurements and a concentration argument for honest-prover liveness.

Sources

• ADPS16 / NewHope, USENIX Security 2016
• Pinned lattice-estimator reduction model
• PR #343

Reviewer map

Suggested reading order:

1. crates/akita-sis-estimator/src/euclidean.rs
2. crates/akita-types/src/sis/ajtai_key.rs
3. crates/akita-types/src/sis/norm_bound.rs
4. crates/akita-planner/src/schedule_params/candidate/recursive.rs
5. crates/akita-planner/src/schedule_params/suffix_dp.rs
6. crates/akita-prover/src/protocol/core/fold/mod.rs
7. crates/akita-verifier/src/protocol/core/fold/mod.rs
8. crates/akita-types/src/proof/wire.rs
9. crates/akita-planner/src/schedule_params/tests.rs

[cursor]

PR Summary

High Risk
Changes SIS security pricing, fold scheduling, and proof transcripts around an intentionally unchecked L² claim on security-critical commitment parameters; safe only as a labeled non-production diagnostic.

Overview
This is an explicit unsound diagnostic — it prices some late single-chunk folds with a whole-witness L² cap and ships the prover’s measured squared norm on the wire, but the verifier only shape-checks and transcript-binds it; there is no norm sumcheck.

SIS / tables: Replaces the retired BDGL16 Euclidean path with 128-bit quantum ADPS16 pricing and a new generated_l2_sis_table. The estimator now treats the table key as the full scalar collision vector’s squared L² norm (length_bound = sqrt(collision_l2_sq)), fixing the prior width double-count. InnerCommitMatrixParams can carry unchecked_l2_collision_sq and be built via try_new_with_unchecked_l2_diagnostic_min_rank; validate() audits that path separately from ordinary L∞ floors.

Planner: Late folds (level ≥ 3, single chunk) may emit multiple inner-commit candidates — L∞ plus widening L² buckets — and derive_candidate_level_params_frontier keeps the best split per A output rank instead of a single overwritten choice. Direct terminal suffix rejects schedules whose terminal witness still has the L² diagnostic flag. Catalog emission and schedule expansion thread unchecked_l2_collision_sq through generated folds.

Proof / prover / verifier: Introduces UncheckedL2NormDiagnostic on FoldLevelProof (claimed u128 norm + one placeholder challenge coefficient per final stage-1 round), updates shapes, wire serde, and level_proof_bytes. The prover accumulates exact folded-witness L² during ring-switch and absorbs the diagnostic into the transcript; optional AKITA_L2_DIAGNOSTIC_GRIND_START varies fold-grind nonces for profiling. Config schedule width audit for A-rows delegates to inner_commit_matrix.validate().

Tooling: PCS test shapes and profile reporting account for diagnostic bytes; planner tests document updated proof-payload byte expectations and guard that L² candidates cannot terminate directly.

^{Reviewed by Cursor Bugbot for commit be2fbe5. Bugbot is set up for automated code reviews on this repo. Configure here.}

[quangvdao]

Planner-frontier fix and security-model correction are now on the final head be2fbe5c4.

The planner bug was that recursive search kept only the locally cheapest A matrix. It now keeps one best continuation per distinct secure A rank, plus the ordinary L-infinity fallback, so suffix DP can expose genuine fold and tail changes.

During the audit I found that the first L2 experiment used coefficient ||c||₂ where ring multiplication requires the negacyclic-convolution operator norm Γ = op_norm(c). The corrected collision price is:

C_A = 64 · Γ² · E · ||z||₂²

where E = 1 for q128 and E = 2 for the current q64/q32 paired-lane embedding.

At diagnostic Γ = 17 and ||z||₂² <= 2³², the final D128 one-hot results are:

Profile	PR #343	Corrected draft	Delta	Terminal blocks	Fold count
fp64	83,602 B	83,716 B	+114 B	7	unchanged
fp32	77,831 B	75,842 B	-1,989 B	6	unchanged

The earlier five-block tails at fp64 75,929 B and fp32 73,136 B came from the invalid coefficient-L2 substitution. They are only an optimization upper bound, not a security result. Reaching that 2^44 small-field collision bucket with a 2^32 norm cap would require integer Γ <= 5, while Parseval already gives Γ >= sqrt(31) ≈ 5.57, so that path cannot accept a challenge in the current shell.

All local validation listed in the revised PR body passes. Exact-head GitHub CI is still running.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit be2fbe5. Configure here.}

[cursor]

L2 validate skips key audit

Medium Severity

When unchecked_l2_collision_sq is set, validate returns after only checking the L2 rank floor. It never runs audit_commit_matrix_fields, so stale table_digest, mismatched sis_table_key.role, and other key-identity checks are skipped. The schedule SIS audit now relies on this path for A matrices, so malformed L2-marked keys can pass where ordinary matrices would fail. That gap is separate from the intentional missing norm relation check.

Additional Locations (1)

• crates/akita-config/src/lib.rs#L577-L584

 

^{Reviewed by Cursor Bugbot for commit be2fbe5. Configure here.}

[quangvdao]

Closing this unchecked diagnostic. It is superseded by the sound selective L2 design: late nonterminal folds may add a verified L2 route while every fold retains the existing L-infinity route. I will link the replacement draft here once it is open.

[quangvdao]

Replacement draft: #369. It starts from current main and implements the sound selective design rather than the unchecked global diagnostic.