Repository navigation
perf(javascript): reduce retained semantic analysis allocations - #1601
Conversation
|
Independent review of |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
…3' into perf/javascript-semantic-ids-1563
|
Final-head CI at
These failures concern files outside this seven-file allocation change. The independent final-head semantic and real large-module CLI/MCP results in the PR description remain tied to their stated revisions. I am tracking the maintenance failures rather than bypassing the gate; this PR must remain unmerged until all checks on its eventual final head pass. CI: https://github.com/morluto/rea/actions/runs/38073016162/job/114275323749 |
Large JavaScript bundles retain concatenated semantic IDs, empty collections for every mutation observer, and an eager scope index for every AST node. Construct high-volume IDs with one join, materialize observer collections only when they acquire facts, and resolve lexical scopes through explicit boundaries and the existing AST parent index. Cache traversed ancestry on demand so repeated scope lookup remains linear in the controlled depth case.
Refs #1065 and #1563. This is an allocation reduction: no new source cutoff, skipped facts, result cap, schema, tool or authority change. The full-directory OOM remains; #1563 must stay open. The PR diff remains seven files and preserves the original implementation and independent cache follow-up history.
Validation:
a2255f49integrates main16d9c5ad, including the existing fix(ci): preserve cleanup boundaries and bounded failure assertions #1618 cleanup/CI repair. Fresh runtime preparation, 85 affected browser/process-output/Qwen/EVM/firmware cases across seven files (2 host-only skips), andnpm run check:fastpass. The preceding integration30a33f92passed 495 semantic cases across 20 files plus build/static checks; the seven changed files are identical to measured head03994825.fa85af8e: the depth regression fails before the ancestry cache and passes after it. At depths 80/160/320, AST-parent reads change from 7,161/27,081/105,321 to 332/652/1,292. Its verification reports 494 semantic tests across 21 files, all static/documentation/architecture/discovery checks, four unchanged complete-IR controls, and a real 674-byte public CLI/pinned-SDK MCP control preserving Evidence IDs, text/structured parity, source bytes and subsequent ping. These structural counts do not establish general throughput or billed-token improvements.03994825: 4,762,637 bytes, SHA25664516393b8f06dce5b74140781d43b1df4e743de3e283526a533508ca1c58147. Production domain analysis completes with 768 MiB old-space, retaining the complete 256,300,420-byte IR, SHA256c109f190e1fb1719b73705096730bf4876338d191a9a07d88e3f6eaed019df04. Peak family RSS 1,006,104,576 bytes. Thirty additional complete-IR corpus digests matched the original source owners before the later integrations. Domain equality does not establish full public CLI success at that heap size.03994825, parent 512/server 1024 MiB: complete 170-tool catalog; the completed member returns a typed oversized-transport constraint with retained Evidence. Summary, module page, exact node selection, one-node incoming trace (one match), ping, close/stale-reference and subsequent small analysis pass. Source SHA unchanged; Evidenceev_c11a23f39bc7f3f371eeea1c5754dbc2d3923b124b2ac83562c73112f27c87a9; peak family RSS 1,656,877,056 bytes. The application graph explicitly retains its existing ceiling/partial semantic coverage. These public proofs remain tied to the stated frozen revision.30a33f92, including maindce81ca4and all seven allocation/cache changes: still fatal heap exhaustion atchunk-kasbfbhj.js(1432/2351), exit 134 at 1024 MiB, zero CLI stdout, 177.39 seconds, peak family RSS 1,362,915,328 bytes. The external guard did not stop it. All 2589 extracted file digests were independently rechecked: zero mismatches. No causal speedup follows from single runs at different revisions.Linux x64, Node 24.18.0/npm 11.16.0, CPUs 13/15, nice 10, one heavy lane, 2304 MiB process-family RSS guard and 2 GiB host reserve. Real Hopper/Ghidra and Windows/macOS verification were not performed for this JavaScript-only change. Final-head CI is pending; merge requires all checks on the exact final head to pass.
Separate follow-up: #1610's fixed source cutoff was reviewed through real full-directory CLI/MCP and a below-cutoff synthetic module and still permits host termination: #1610 (review) . No cutoff or worker protocol is included here. My overlapping test-only #1617 was closed as superseded by #1618; it adds no files to this PR.