fix(javascript): isolate source analysis and retain bounded partial evidence

.agents/skills/reverse-engineer-anything/references/javascript-applications.md

@@ -9,6 +9,22 @@ that static analysis. Read findings, coverage, limitations and unknowns, then
 follow a specific unanswered question. Older servers without `detail` return
 the complete application Evidence inline when it fits.
 
+When the connected schema advertises `max_heap_mb` and `analysis_timeout_ms`,
+these select the isolated source analyzer's old-space heap and per-source
+deadline. Defaults are 1024 MiB and 300000 ms. They do not change the MCP
+process heap. Resource-limited analyses preserve completed static facts and
+report partial semantic coverage with specific unknown scopes; this does not
+mean the source is malformed. Inspect those limits before retrying a source.
+When expanded relationships exceed the parent's available memory, inspect the
+file's modules item view for retained `static_analysis` and
+`semantic_module_analysis`. Their facts remain available even when graph links
+are unknown. Cumulative result exhaustion can stop analysis before every file
+is examined; the inventory and remaining unknown scopes distinguish that case.
+Timeout or cancellation after source analysis begins retains partial Evidence
+in the same session. A returned `error.details.partial_observation` reference
+can be supplied directly to `inspect_analysis_view`. If the SDK cancels locally, retrieve its recorded ID from
+the Evidence bundle before inspecting the partial analysis.
+
 ## Read a module without repeating analysis
 
 Use the actual returned parent ID, on the same connection, as `source.evidence_id`.

docs/javascript-artifact-reconstruction.md

@@ -89,9 +89,9 @@ queries use per-file range indexes, while call sites use exact-range lookups.
 Existing node budgets and coverage reporting remain in effect.
 
 Progress identifies the source file being parsed and projected. Cancellation is
-checked between files, with event-loop yields that allow notifications and
-completed file-local allocations to be released. A single file's synchronous
-parser and semantic extraction still need to finish before that boundary.
+checked between files and throughout result transfer. The owned analyzer can
+be stopped while its synchronous parser or semantic extractor is active;
+the CLI/MCP process remains available for control requests.
 
 Graph and Evidence identifiers hash canonical JSON incrementally, without
 assembling a single string for the whole graph. The canonical bytes and existing
@@ -113,6 +113,46 @@ smaller view, for example `--format json --filter-output
 evidence_id,normalized_result.statistics`. Streaming output does not bound the
 memory needed to construct the analysis graph itself.
 
+JavaScript source parsing, static analysis and semantic projection run in one
+owned analysis process, reused sequentially across files. Target code is never
+executed. The analyzer defaults to 1024 MiB of V8 old-space and a 300000 ms
+deadline per source, including startup and fact transfer. Select
+`max_heap_mb` and `analysis_timeout_ms` through MCP, or `--max-heap-mb` and
+`--analysis-timeout-ms` on the dedicated CLI command. These choices control the
+analyzer; the caller's `NODE_OPTIONS` still controls the CLI/MCP process heap.
+V8 old-space limits do not impose a process RSS limit.
+
+A source that exhausts the analyzer heap does not terminate the CLI/MCP
+process. Completed static facts are retained, other files can still be analyzed,
+and unexamined semantic relationships remain explicit unknowns. The parent
+also checks transfer expansion and accumulated results against its remaining
+heap before accepting semantic facts. A resource-limited result has partial
+coverage, the observed limits and an `analyze-javascript-source` unknown scope
+with the specific failure, including captured diagnostics when available.
+Resource failures do not increment syntax parse failures. There is no fixed
+source-length cutoff.
+
+Static observations also need memory when expanded into nodes, relationships
+and Evidence. If that expansion does not fit, the file node retains
+`static_analysis` and any admitted `semantic_module_analysis` observations,
+including their exact values and source locations. Use the public modules item
+view to read them. The graph marks those unexpanded relationships as unknown.
+If cumulative result capacity is exhausted, analysis stops; inventoried but
+unexamined files remain present with explicit unknown scopes.
+
+A timeout stops the remaining analysis and returns typed failure details with
+completed partial Evidence. MCP failure details return
+`partial_observation: { kind: "retained-evidence", evidence_id: "..." }`
+after recording succeeds, so `inspect_analysis_view` can open that analysis
+directly without serializing the entire partial graph into the error response.
+CLI failure output retains the complete partial Evidence.
+Cancellation after source analysis begins likewise
+retains completed facts; an SDK client may reject its cancelled request before
+receiving that reply. On the same MCP connection, inspect the Evidence bundle
+and use the retained analysis ID with `inspect_analysis_view`. Owned analyzer
+cleanup is verified before its private files are released. Cleanup uncertainty
+is reported separately and retains ownership for retry.
+
 CLI workflows parse larger JSON input files incrementally from one verified
 regular-file handle, without constructing a document-sized string. Smaller
 files retain native JSON parsing. Both paths require strict JSON and valid

knip.json

@@ -3,6 +3,7 @@
   "entry": [
     "src/main.ts",
     "src/cli.ts",
+    "src/composition/javascriptAnalysisWorkerProcess.ts",
     "src/contracts/errorSchemas.ts",
     "src/evaluation/AgentEvaluationFixtures.ts",
     "src/application/RealHopperAssertions.ts",

package.json

@@ -30,6 +30,9 @@
     "access": "public"
   },
   "type": "module",
+  "imports": {
+    "#javascript-analysis-worker": "./dist/composition/javascriptAnalysisWorkerProcess.js"
+  },
   "bin": {
     "rea": "scripts/rea.mjs",
     "rea-agents": "scripts/rea.mjs"

src/application/javascript/JavaScriptApplicationService.ts

@@ -6,14 +6,9 @@ import {
   analyzeJavaScriptApplicationInputSchema,
   parseOwnedJavaScriptApplicationAnalysisSteps,
 } from "../../domain/javascript/javascriptApplicationAnalysis.js";
-import {
-  AnalysisAccessDeniedError,
-  AnalysisInputError,
-  AnalysisOutputError,
-  AnalysisUnsupportedTargetError,
-} from "../../domain/analysisErrorCore.js";
+import { AnalysisOutputError } from "../../domain/analysisErrorCore.js";
 import { ArtifactOperationError } from "../../domain/artifactOperationError.js";
-import { type AnalysisError } from "../../domain/analysisErrorBase.js";
+import { AnalysisError } from "../../domain/analysisErrorBase.js";
 import type { Evidence } from "../../domain/evidence.js";
 import { analysisInputErrorFromIssues } from "../../domain/inputIssueProjection.js";
 import { err, ok, type Result } from "../../domain/result.js";
@@ -64,6 +59,12 @@ export const analyzeJavaScriptApplicationValidated = async (
         input_path: input.input_path,
         format: input.format,
         integrity_policy: input.integrity_policy,
+        ...(input.max_heap_mb === undefined
+          ? {}
+          : { max_heap_mb: input.max_heap_mb }),
+        ...(input.analysis_timeout_ms === undefined
+          ? {}
+          : { analysis_timeout_ms: input.analysis_timeout_ms }),
       },
       scope,
       options.signal,
@@ -110,13 +111,8 @@ export const analyzeJavaScriptApplicationValidated = async (
     assertJavaScriptAnalysisActive(options.signal);
     return ok(evidence);
   } catch (cause: unknown) {
-    // Input selection failures are already typed by their cause.
-    if (
-      cause instanceof AnalysisInputError ||
-      cause instanceof AnalysisUnsupportedTargetError ||
-      cause instanceof AnalysisAccessDeniedError
-    )
-      return err(cause);
+    // Preserve execution constraints and their completed partial Evidence.
+    if (cause instanceof AnalysisError) return err(cause);
     if (cause instanceof ArtifactReaderFailure)
       return err(artifactFailureToOperationError(cause));
     if (cause instanceof z.ZodError)

src/application/javascript/JavaScriptArtifactAnalysis.ts

@@ -8,6 +8,7 @@ import {
   classifyParsedJavaScriptOpenReceiversSteps,
 } from "../../domain/javascript/javascriptSemanticAnalysis.js";
 import type { JavaScriptSemanticIr } from "../../domain/javascript/javascriptSemanticIr.js";
+import { projectJavaScriptModuleSemantics } from "../../domain/javascript/javascriptModuleSemanticIr.js";
 import { parseJavaScriptSource } from "../../domain/javascript/javascriptSourceParser.js";
 import { hasValidSourceMapContents } from "../../domain/sourceMapContents.js";
 import { flattenSourceMapLeaves } from "../../domain/sourceMapEnvelope.js";
@@ -35,6 +36,15 @@ import type {
 import { analyzeJavaScriptJsonModule } from "./JavaScriptJsonModules.js";
 import { completeJavaScriptAnalysisSteps } from "./JavaScriptAnalysisControl.js";
 import { htmlArtifactReferences } from "../../domain/javascript/htmlArtifactReferences.js";
+import type { JavaScriptSourceAnalysisPort } from "../../domain/javascript/javascriptSourceAnalysis.js";
+import type { JavaScriptSemanticGraphProjection } from "./JavaScriptSemanticGraphBuilder.js";
+import { AnalysisError } from "../../domain/analysisErrorBase.js";
+import { projectAnalysisError } from "../../domain/analysisErrorProjection.js";
+import {
+  AnalysisCancelledError,
+  AnalysisResourceConstraintError,
+  AnalysisTimeoutError,
+} from "../../domain/analysisErrorCore.js";
 
 interface MutableArtifactAnalysis<
   SemanticIr extends JavaScriptModuleSemanticIr,
@@ -95,27 +105,7 @@ export const analyzeAndProjectJavaScriptArtifactFiles = async (
   ): Generator<void, JavaScriptModuleSemanticIr> {
     const projection = projectSemantics(file, ir);
     if (projection !== undefined) while (projection.next().done !== true) yield;
-    const programScopes = ir.scopes.filter(({ kind }) => kind === "program");
-    const programScopeId = programScopes[0]?.scopeId;
-    const localNames = new Set(
-      ir.moduleLinks.map(({ localName }) => localName),
-    );
-    const callableIds = new Set(
-      ir.moduleLinks.map(({ callableId }) => callableId),
-    );
-    return {
-      scopes: programScopes,
-      bindings: ir.bindings.filter(
-        ({ name, scopeId }) =>
-          scopeId === programScopeId && localNames.has(name),
-      ),
-      callables: ir.callables.filter(({ callableId }) =>
-        callableIds.has(callableId),
-      ),
-      moduleLinks: ir.moduleLinks,
-      coverage: ir.coverage,
-      limitations: ir.limitations,
-    };
+    return projectJavaScriptModuleSemantics(ir);
   };
   const context = { state, projectSemantics: projectFileSemantics };
   for (const [index, file] of fileSet.files.entries()) {
@@ -130,6 +120,174 @@ export const analyzeAndProjectJavaScriptArtifactFiles = async (
   return finalizeArtifactAnalysis(state);
 };
 
+/** Analyze sources behind a hard heap boundary, retaining collected facts on interruption. */
+export const analyzeAndAdoptJavaScriptArtifactFiles = async (
+  fileSet: JavaScriptArtifactFileSet,
+  worker: Pick<JavaScriptSourceAnalysisPort, "analyze">,
+  projection: JavaScriptSemanticGraphProjection,
+  beforeFile: (
+    file: JavaScriptArtifactFile,
+    completed: number,
+    total: number,
+  ) => Promise<void>,
+  signal?: AbortSignal,
+): Promise<{
+  readonly analysis: JavaScriptModuleArtifactAnalysis;
+  readonly interruption: AnalysisError | null;
+}> => {
+  const state = emptyArtifactAnalysis<JavaScriptModuleSemanticIr>();
+  let interruption: AnalysisError | null = null;
+  for (const [index, file] of fileSet.files.entries()) {
+    try {
+      await setImmediate();
+      await beforeFile(file, index, fileSet.files.length);
+      if (signal?.aborted === true)
+        throw new AnalysisCancelledError("analyze_javascript_application");
+      if (file.kind !== "javascript" || !file.text.included) {
+        const steps = analyzeArtifactFileSteps(file, {
+          state,
+          projectSemantics: function* (_file, ir) {
+            yield;
+            return projectJavaScriptModuleSemantics(ir);
+          },
+        });
+        await completeJavaScriptAnalysisSteps(steps, signal);
+        continue;
+      }
+      const result = await worker.analyze(
+        file,
+        projection.remainingFileNodeBudget(),
+        signal,
+      );
+      const facts = result.ok ? result.value : result.error;
+      const applicationProjectionFailure = facts.applicationProjectionFailure;
+      if (facts.projection !== null)
+        projection.adoptFileProjection(facts.projection);
+      const error = result.ok ? null : result.error.error;
+      const limits =
+        error instanceof AnalysisResourceConstraintError
+          ? error.reportedLimits
+          : error instanceof AnalysisTimeoutError
+            ? { worker_timeout_ms: error.timeoutMs }
+            : null;
+      const failure =
+        error === null
+          ? undefined
+          : {
+              reason: `${file.path}: ${error.message}`,
+              error: projectAnalysisError(error),
+              limits,
+            };
+      state.files.push({
+        file,
+        javascript: facts.javascript,
+        semantic: facts.module === null ? null : { ir: facts.module },
+        ...(failure === undefined ? {} : { analysis_failure: failure }),
+        ...(applicationProjectionFailure === undefined
+          ? {}
+          : {
+              application_projection_failure: {
+                reason: applicationProjectionFailure.message,
+                error: projectAnalysisError(applicationProjectionFailure),
+                limits: applicationProjectionFailure.reportedLimits,
+              },
+            }),
+      });
+      if (failure !== undefined && facts.projection === null)
+        projection.recordUnavailableFile(
+          file,
+          failure.reason,
+          failure.limits,
+          error instanceof AnalysisResourceConstraintError ||
+            error instanceof AnalysisTimeoutError
+            ? "resource-limit"
+            : "incomplete-module",
+        );
+      if (facts.javascript !== null) {
+        state.visitedNodes += facts.javascript.visited_ast_nodes;
+        state.findings +=
+          findingCount(facts.javascript) +
+          (facts.module?.moduleLinks.length ?? 0);
+        state.modules += facts.javascript.bundler_registrations.reduce(
+          (count, item) => count + item.modules.length,
+          0,
+        );
+        if (facts.javascript.parse_status === "failed")
+          state.parseFailures += 1;
+      }
+      if (
+        error !== null &&
+        (!(error instanceof AnalysisResourceConstraintError) ||
+          error.reportedLimits?.parent_result_budget_exhausted === true)
+      ) {
+        interruption = error;
+        break;
+      }
+    } catch (cause: unknown) {
+      if (signal?.aborted !== true && !(cause instanceof AnalysisError))
+        throw cause;
+      interruption =
+        signal?.aborted === true
+          ? new AnalysisCancelledError("analyze_javascript_application")
+          : cause instanceof AnalysisError
+            ? cause
+            : null;
+      if (interruption !== null) {
+        state.files.push({
+          file,
+          javascript: null,
+          semantic: null,
+          analysis_failure: {
+            reason: `${file.path}: ${interruption.message}`,
+            error: projectAnalysisError(interruption),
+            limits: null,
+          },
+        });
+        projection.recordUnavailableFile(
+          file,
+          interruption.message,
+          null,
+          "incomplete-module",
+        );
+      }
+      break;
+    }
+  }
+  if (interruption !== null) {
+    const analyzedPaths = new Set(state.files.map(({ file }) => file.path));
+    const failure = {
+      reason: `Source analysis was interrupted: ${interruption.message}`,
+      error: projectAnalysisError(interruption),
+      limits:
+        interruption instanceof AnalysisResourceConstraintError
+          ? interruption.reportedLimits
+          : interruption instanceof AnalysisTimeoutError
+            ? { worker_timeout_ms: interruption.timeoutMs }
+            : null,
+    };
+    for (const file of fileSet.files) {
+      if (analyzedPaths.has(file.path)) continue;
+      state.files.push({
+        file,
+        javascript: null,
+        semantic: null,
+        analysis_failure: failure,
+      });
+      if (file.kind === "javascript")
+        projection.recordUnavailableFile(
+          file,
+          failure.reason,
+          failure.limits,
+          interruption instanceof AnalysisResourceConstraintError ||
+            interruption instanceof AnalysisTimeoutError
+            ? "resource-limit"
+            : "incomplete-module",
+        );
+    }
+  }
+  return { analysis: finalizeArtifactAnalysis(state), interruption };
+};
+
 const analyzeArtifactFiles = <SemanticIr extends JavaScriptModuleSemanticIr>(
   fileSet: JavaScriptArtifactFileSet,
   projectSemantics: ArtifactAnalysisContext<SemanticIr>["projectSemantics"],
@@ -161,6 +319,12 @@ const finalizeArtifactAnalysis = <
     limitations: [
       "JavaScript and HTML were parsed as inert text; bundle bootstrap code was never executed.",
       "Static paths and relationships may remain unresolved when expressions are dynamic or obfuscated.",
+      ...state.files.flatMap(
+        ({ analysis_failure, application_projection_failure }) =>
+          [analysis_failure, application_projection_failure].flatMap(
+            (failure) => (failure === undefined ? [] : [failure.reason]),
+          ),
+      ),
     ],
   };
 };