Skip to content

fix(javascript): keep static facts after semantic results use the parent budget - #8648

Merged
kaoru0822-kitauji merged 2 commits into
mainfrom
fix/7736-javascript-static-fact-budget
Nov 2, 2026
Merged

kaoru0822-kitauji merged 2 commits into
mainfrom
fix/7736-javascript-static-fact-budget

Conversation

@kaoru0822-kitauji

Copy link
Copy Markdown
Collaborator

What broke: Once semantic and application charges used the parent's result budget, the next static checkpoint stopped the analysis, so every remaining file lost its static facts while most of the parent heap was free.
The fix: Static checkpoints may now continue past that budget, up to three quarters of the free heap, while the live heap also has room for another copy of what it holds. Semantic and application results still stop at the budget.
The risk: More facts are retained, so peak parent memory rises when a run now completes (mixed fixture peak heap about 1.3 GiB to 2.4 GiB, limit 4.2 GiB). The three-quarter share is a judgment from runs on synthetic fixtures, not a proven bound for every input shape.

Summary

Fixes #7736

The issue's exact 301-file recipe no longer fails on latest main: #7744 and #7986 cover it, and it exited 0 with 301/301 files in 4 runs. The root cause is still there for any input whose earlier charges cannot be reclaimed. A fixture with vendor-style chunks first still exits 1, with 90 to 98 of 501 files losing their static facts.

Problem and expected behavior

JavaScriptAnalysisWorker.#readCheckpoint admitted static checkpoints against the same budget as semantic and application results, half of the parent's free heap. #7986 made application reservations reclaimable, but adopted semantic results are not. Once they fill the budget, the next checkpoint exhausts it.

Observed on 8250bf232 with the generator below (501 files, 13,323,141 bytes), --max-heap-mb 4096, default parent heap:

{"code":"resource_constraint","message":"The parent's result capacity was exhausted after retaining static facts for chunk-0208.js; remaining sources were not analyzed"}
  • parsed_javascript_files was 409, 411 and 403 of 501 across three runs, so the failure point moves.
  • Sampled parent heap peaked at 1.2 to 1.3 GiB against a 4.2 GiB limit.
  • 191 to 199 files also lost their application relationships to reclamation that did not save the run.

Expected: files the parent can hold keep their static facts, and the run completes with partial semantic coverage reported explicitly.

Change and scope

All admission changes are in #readCheckpoint and #refreshResultBudget, so the CLI and MCP share them.

  • Semantic and application offers are unchanged and still stop at half of the free heap.
  • Static checkpoints may continue to three quarters of it. Reservations are reclaimed only when that larger share runs short.
  • Past the budget, a checkpoint also needs retained + used <= limit - used - transfer reserve, meaning the live heap has room for another copy of what it holds. Without that room, admission falls back to the old budget.
  • The exhaustion failure now reports the limit that applied, what was available, the live heap in use and what the checkpoint needed.

The live condition is required. A first version admitted checkpoints up to the whole free-heap figure and the parent aborted with SIGABRT at --max-old-space-size=160 and 192, where base stops with a typed failure. A three-quarter share alone still aborted at 160. V8's heap_size_limit includes young-generation capacity (limit 384 MiB at old space 192 MiB; the abort came at about 186 MiB), so the existing live check does not protect the parent near its real limit. This PR does not correct that figure for the result budget.

Not included: the later Evidence stages listed under proof gaps, and #3671. The projection refusal path it reports is unchanged. #8411 and #8468 are untouched.

Contract and boundary impact

  • Semantic owner and earliest changed stage: JavaScriptAnalysisWorker owns parent result admission. Its consumer, analyzeAndAdoptJavaScriptArtifactFiles, is unchanged. No superseded path was removed: reclamation and the next-checkpoint reserve still apply.
  • Contribution scope fit and linked maintainer decision: an ordinary fix within an existing boundary, no new provider or capability. The issue calls separate static admission a design decision for maintainers, so this PR is a proposal for that decision. The matching merge checkbox is left unticked.
  • CLI and MCP/tool-catalog contract: no schema or catalog change. reported_limits on the exhaustion failure gains parent_static_capacity_bytes, parent_available_static_capacity_bytes, parent_used_heap_bytes and static_retention_estimate_bytes. These are not added to coverage limits.
  • Provider, bridge, target-format, or platform compatibility: no provider or bridge change. Bun was not exercised.
  • Evidence, artifact, provenance, or reconstruction contract: no shape change. More files carry static facts, and refused relationships stay reported through the existing application_projection_failure.
  • Target selection, authority, upstream behavior, and cleanup details: not applicable.
  • Generated metadata, package, or installation impact: none.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated:
    • tests/acceptance/applications/javascriptCheckpointReserveCli.test.ts: a public CLI run with a real tight parent heap examines all 17 files, and each later source keeps its static facts and reports the omission.
    • tests/boundary/javascript/javascriptStaticProjectionReclamation.test.ts: three new worker cases (past the budget, unreclaimable semantic charges, no live room) and updated thresholds.
  • Failure on base: run against the base worker, 6 of these tests fail, including the CLI test with the exhaustion error. All pass with the fix.
  • User-visible output: the fixture above now exits 0 with parsed_javascript_files: 501, skipped_javascript_files: 0. A real stop now reports (excerpt):
{"parent_result_budget_bytes":103928540,"parent_static_capacity_bytes":103928540,"parent_available_static_capacity_bytes":5710,"parent_used_heap_bytes":197340816}

Before and after, executed. CLI stdout, --max-heap-mb 4096, default parent heap (limit 4288 MiB), two runs each on macOS with Node 24.22.1. The machine was shared with other jobs, and the base and fix batches ran separately, so small timing differences are not meaningful.

Fixture Build Exit Static facts Relationships reclaimed Elapsed s CPU s Peak family RSS MiB Peak parent heap MiB stdout MB
Issue recipe, 301 files base 0, 0 301, 301 3, 3 61.3, 63.8 58.7, 72.5 2266, 2255 1914, 1938 307.3, 307.7
Issue recipe, 301 files fix 0, 0 301, 301 0, 0 46.0, 46.2 51.7, 51.5 2318, 2289 1936, 1880 311.5, 311.5
Mixed, 501 files base 1, 1 411, 403 199, 191 74.0, 53.0 73.4, 56.0 1533, 1652 1362, 1373 262.8, 263.6
Mixed, 501 files fix 0, 0 501, 501 0, 0 68.0, 66.6 73.8, 72.9 3152, 3065 2508, 2385 511.3, 511.0

The mixed rows are not equivalent work: base stops early and fix completes. Cold and warm:

  • CLI runs are separate processes. A source search found no result cache on this path, and the filesystem cache was not controlled, so run 1 and run 2 are repeats and not a true cold/warm pair.
  • The only warm path measured is a second MCP call on one server (below).
  • Heap was sampled every 100 ms and can miss peaks in synchronous work. Tokens were not measured.

Parent heap sweep, executed. Issue generator at 901 files, one run per cell. Files with static facts:

--max-old-space-size 128 160 192 224 320 512 1024
base SIGABRT 102 160 211 295 296 451
fix SIGABRT 102 161 210 294 624 901

Every run from 160 to 512 exits 1 without aborting on both builds. At 1024, base exits 1 and fix exits 0. The abort at 128 happens at about 4 s on both builds. One larger fix-only run, 3,051 files and 81 MB at the default heap, exited 0 with 3051/3051 in 214.8 s, peak heap 2821 MiB, RSS 3389 MiB, stdout 658.9 MB.

Remaining proof gaps:

  • MCP does not deliver a result on these fixtures, on either build. Over real stdio, analyze_javascript_application on the 201-file, 301-file and mixed fixtures ends in an error. On fix it is always record_evidence, "The investigation's retained representation allowance is exhausted". Base gives the same error on 201 and 301 files. On mixed, base fails earlier with the compact response-budget error. So MCP callers do not get these facts until that allowance is addressed. Two calls on one server for mixed took 72.9 s and 63.1 s on base, 77.2 s and 51.2 s on fix, peak RSS 2624 and 2647 MiB against 3000 and 2746 MiB, all ending in those errors.
  • --detail summary --evidence_output still exits 1 on these fixtures with "Insufficient heap headroom to authenticate the Evidence checkpoint" ([Bug] analyze-javascript-application on GitHub Desktop 3.6.5 fails at the final Evidence checkpoint and keeps nothing #8411). Mixed on base stops earlier at the static stage.
  • Synthetic fixtures only. No real application, real provider, Windows, Linux or Bun run. CI covers other platforms.
  • used_heap_size includes uncollected garbage, so the live condition can fall back to the old budget early. That outcome matches base. Semantic coverage can still vary slightly between runs.
  • No default-heap run reached the three-quarter share, and I did not establish that any real run stopped on it. In the one stop I inspected (320), the live condition set the limit. Only the mocked worker tests pin the share itself.

Boundary regressions: public CLI workflow plus production-boundary worker tests, covering capacity and partial-coverage reporting. Lifecycle regressions do not apply: no process or resource ownership changed.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.
gen-mixed.mjs (deterministic synthetic fixture: node gen-mixed.mjs ./fx 200 300)
// Vendor-style chunks (plain functions, large semantic graphs, few findings) sort first,
// then chunks from the issue's generator shape (many storage/request/import findings).
// Usage: node gen-mixed.mjs <outDir> <vendorCount> <appCount>
import { mkdirSync, writeFileSync, rmSync } from "node:fs";
import { join } from "node:path";
const outDir = process.argv[2];
const vendorCount = Number(process.argv[3] ?? 40);
const appCount = Number(process.argv[4] ?? 200);
rmSync(outDir, { recursive: true, force: true });
const assets = join(outDir, "assets");
mkdirSync(assets, { recursive: true });
let seed = 7;
const rand = () => ((seed = (seed * 1103515245 + 12345) % 2147483648) / 2147483648);
const vendor = Array.from({ length: vendorCount }, (_, i) => `a-vendor-${String(i).padStart(3, "0")}.js`);
const app = Array.from({ length: appCount }, (_, i) => `chunk-${String(i).padStart(4, "0")}.js`);
const fill = (target, make) => { const parts = []; let n = 0; let size = 0; while (size < target) { const fn = make(n); parts.push(fn); size += fn.length; n++; } return parts.join(""); };
let total = 0;
const write = (name, text) => { writeFileSync(join(assets, name), text); total += text.length; };
vendor.forEach((name, i) => write(name, `import{helper}from"./index.js";\n` + fill(20 * 1024 + Math.floor(rand() * 12 * 1024), (n) => `export function v${i}_${n}(a,b){const c=a*${n}+b;if(c>${Math.floor(rand() * 1000)}){return helper(c)+${n};}const d=helper(a);return d-c;}\n`)));
app.forEach((name, i) => write(name, `import{api}from"./index.js";\nexport const load${i}=()=>import("./${app[(i + 1) % appCount]}");\n` + fill(20 * 1024 + Math.floor(rand() * 12 * 1024), (n) => { const key = `demo.pref.${i}.${n % 5}`; return `export function f${i}_${n}(a,b){const c=a*${n}+b;if(c>${Math.floor(rand() * 1000)}){localStorage.setItem("${key}",String(c));}const d=localStorage.getItem("${key}");return api.rpc("proc_${i}_${n % 7}",{v:c,d}).then(r=>r.data);}\n`; })));
const entry = `export const helper=(x)=>x+1;\nexport const api={rpc(n,a){return fetch("/rpc/"+n,{method:"POST",body:JSON.stringify(a)}).then(r=>r.json());}};\n` + [...vendor, ...app].map((n, i) => `export const route${i}=()=>import("./${n}");\n`).join("");
write("index.js", entry);
console.log(JSON.stringify({ files: vendorCount + appCount + 1, bytes: total }));

Validation performed

  • npm run check:fast — exit 0, on the final tree
  • npm run format:check — exit 0, on the final tree
  • npm run docs:check — exit 0, on the final tree
  • npm run test:focused on the four budget and reclamation test files — 17 of 17 passed, on the final tree
  • npm run test:focused on 136 JavaScript, Electron and acceptance test files — 1671 of 1671 passed, before merging the latest main commit, which touches only native tracing
  • Real CLI and stdio MCP runs — results above. The measured build has the same source as this branch's worker, before that merge
  • npm run metadata:check — not run, no catalog change

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Exit code changes from 1 to 0 for runs that now complete.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable to Hopper, Ghidra and browsers; macOS only locally.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: analysis stays inert and in the existing worker. Peak parent memory can rise as described above.

Author readiness

Human owner: @kaoru0822-kitauji

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

These boxes were ticked by Claude Code under the owner's standing written instruction for this repository's automation, not after a separate human review of this diff.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

…ent budget

Static checkpoints were admitted against the same budget as semantic and
application results, half of the parent's free heap. Once those charges used
it, the next checkpoint stopped the analysis and every remaining source lost
its static facts, with most of the parent heap still free.

Semantic and application results still stop at that budget. Static checkpoints
may now continue to three quarters of the free heap while the live heap also
has room for another copy of what it holds. Without that room, admission falls
back to the result budget. The live condition is needed because the limit V8
reports counts young-generation capacity that retained objects cannot use:
without it, the parent aborted on heap exhaustion at small heap limits.

Reserved relationships are given up only when the larger share runs short. A
stop still fails with a typed error, which now reports the limit that applied,
what was available, the live heap in use and what the checkpoint needed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaoru0822-kitauji kaoru0822-kitauji added bug Something isn't working track:bugs Small-bug loop track PRs labels Nov 2, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T09:04:43.017717Z 8eba339 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge; no blocking code issue was established.

Summary

This PR lets static facts continue past the parent’s semantic-result budget while keeping separate memory checks.

  • JavaScript analysis keeps static facts from later files when heap room allows.

No actionable new issue or in-scope rule violation was established. The ancestry check passed, but the merge bases differed, so the full PR changes were reviewed. No previous review threads were supplied. Tests were not run during this review. Supersession could not be checked because GitHub access required authentication.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Completed static checkpoint] --> B[Check decode headroom]
  B --> C{Live heap has room for another copy?}
  C -->|Yes| D[Use three-quarter static ceiling]
  C -->|No| E[Use half-capacity result budget]
  D --> F{Checkpoint fits after possible reclamation?}
  E --> F
  F -->|Yes| G[Retain static facts and continue]
  F -->|No| H[Retain completed facts and report exhaustion]
  G --> I[Keep semantic and application offers within result budget]
Loading

Reviews (2) · Last reviewed commit: "Merge origin/main into fix/7736-javascri..." · Reviewed by Greptile

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaoru0822-kitauji
kaoru0822-kitauji merged commit 4a01211 into main Nov 2, 2026
49 checks passed
@kaoru0822-kitauji
kaoru0822-kitauji deleted the fix/7736-javascript-static-fact-budget branch November 2, 2026 13:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working track:bugs Small-bug loop track PRs

Projects

None yet

1 participant