Skip to content

feat(evidence): discover missing Evidence links without transferring records - #8633

Merged
kaoru0822-kitauji merged 9 commits into
mainfrom
feat/8454-evidence-link-inspection
Nov 2, 2026
Merged

kaoru0822-kitauji merged 9 commits into
mainfrom
feat/8454-evidence-link-inspection

Conversation

@kaoru0822-kitauji

@kaoru0822-kitauji kaoru0822-kitauji commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

What broke: an agent could only learn that a retained record's source was absent by transferring complete records or by having a downstream analysis refuse. The summary reported retention: "complete-record" and said nothing about links.
Fix: get_evidence_bundle and CLI inspect-evidence-bundle gain detail: "links", which returns paged link membership with no record payloads, plus optional retained-closure traversal. The summary gains two link counters.
Risk: additive. The summary's output gains two required fields, and default complete delivery is unchanged.

Summary

Fixes #8454 (proposal: #8454 (comment)).

Problem and expected behavior

From the issue: a bundle holding a dossier without its linked source imported cleanly, and the summary showed one complete-record. Only build_call_path revealed the gap, by refusing. detail: "records" reports missing_evidence_links only after transferring the complete records it pages over.

Expected: an agent asks which links are absent before attempting an analysis, without confusing record retention, link membership and analysis coverage, and without filters making an unselected source look absent.

Change and scope

  • inspectEvidenceLinks in src/domain/evidenceRecordSummary.ts: the existing filter owner selects the starting records. Membership is checked against every record in the session or file.
    • traversal: "direct" (default) reads the selected records' own links and sets retained_closure to null.
    • traversal: "retained-closure" also follows retained links, the same walk resolveInvestigationEvidenceClosure performs, and reports complete or incomplete. An empty selection also reports null.
    • Rows are {evidence_id, linked_evidence_id, status} in canonical order. They default to link_status: "missing" and page with offset and limit. The links counts cover every inspected link regardless of that selection.
    • The walk is iterative, and no record payload is read or copied.
  • Summary: direct_links and missing_direct_links, the distinct direct links of the matching records.
  • Adapters: MCP and CLI share inspectEvidenceBundle. A delivery-specific field outside its delivery is refused rather than ignored, and now names its detail. An oversized link page names a smaller selection.
  • Not included: eager traversal on import, rejecting deliberately partial bundles, acquiring missing sources, and any claim about what a missing record contains.

Follow the contribution scope. This reads existing Evidence within existing boundaries, so no maintainer scope decision is required. The API shape was posted on the issue before implementation; there has been no maintainer response yet.

Contract and boundary impact

  • Semantic owner and earliest changed stage: evidenceRecordSummary.ts owns filters, the summary and now links. Downstream refusal stays with resolveInvestigationEvidenceClosure; both read evidence_links as recorded.
  • Contribution scope fit and linked maintainer decision: not applicable, as above.
  • CLI and MCP/tool-catalog contract: a new detail value and two links-only inputs; a new member in the get_evidence_bundle output union; two new required summary fields.
  • Provider, bridge, target-format, or platform compatibility: none; no provider starts.
  • Evidence, artifact, provenance, or reconstruction contract: Evidence and bundle formats are unchanged. Status is membership only; missing means absent from this session or file, not nonexistent.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): none.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: catalog only.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

Before and after, executed. A real Kuna 1.988 session (stdio MCP, compact profile, macOS arm64, Homebrew's unexecuted tmux, SHA-256 8ab1bd9e…a86d, unchanged) ran analyze_function and trace_call_path, then exported a real bundle of 4 records. I removed the linked analyze_function record from the file and imported it into a fresh target-free server. Bytes are the SDK-serialized reply; tokens are tokenx estimates, not billing. This compares whole-session discovery; a caller who already knows the exact record can filter records to fewer bytes.

Question: which sources are absent? Reply bytes Est. tokens Result
Before: detail: "summary" 4,649 (with the new counters) ~1.5k had no link facts on base
Before: detail: "records", whole session 342,601 ~125k one missing ID
After: detail: "links" 1,579 ~500 the same one missing ID, with the record that needs it
After: traversal: "retained-closure" on that record 1,787 ~550 incomplete, 1 of 1 links missing
After importing the complete bundle 1,381 ~440 complete, 0 missing, same Evidence ID

The reported ID was the removed source, and CLI inspect-evidence-bundle on the partial file returned the same rows as MCP.

Scale, source-level. A synthetic linked chain, median of 7 in-process runs, measured as a function call rather than over MCP:

Records Direct, all selected Closure from the head Serialized result
1,000 0.33 ms 0.29 ms about 800 B
10,000 1.92 ms 2.14 ms about 800 B

Schema cost, one catalog per session: compact 580,675 to 581,073 B (+398 B, about +106 estimated tokens); full 3,958,851 to 3,961,803 B (+2,952 B, about +870 estimated tokens).

  • Tests added or updated: tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts (public MCP, full and compact, with every request and reply validated against the definition each profile advertises; removing the links member from the advertised union makes it fail): unselected sources stay retained, no payload in rows, direct versus closure, a record with no links, a repeated link counted once in every delivery, an empty selection, stable paging without skips or duplicates, offset past the end, build_call_path refusing the reported ID and proceeding after import with the original record unchanged, misplaced fields, and overflow naming a smaller page. tests/boundary/cli/analysisViewCli.test.ts covers a partial portable bundle through the shared owner. Four existing tests (three acceptance, one boundary) narrow the wider output union through evidenceBundleSchema, and one verifier fixture gains the new counters.
  • Base reproduction or other evidence: the table above, plus the issue's original observation.
  • User-visible CLI/MCP output (if applicable):
    {"kind":"evidence-link-inspection","traversal":"direct","link_status":"missing","total_retained_records":3,"matching_records":3,"examined_records":3,"links":{"total":1,"retained":0,"missing":1},"retained_closure":null,"matching_links":1,"next_offset":null,"rows":[{"evidence_id":"ev_20c14b56…","linked_evidence_id":"ev_25f85fc5…","status":"missing"}]}
  • Remaining proof gaps: the partial bundle was hand-edited for this test, not produced by a normal exporter. The real session's only links were one view to its parent. Real Ghidra was unavailable on this host, so the build_call_path agreement uses the repository's captured dossier example. The scale runs are one host and a synthetic chain. No autonomous-agent study.

Applicable boundary regressions: identity preservation, page boundaries, omission versus absence. No lifecycle stage changes.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run check:fast — passed
  • npm run format:check — passed
  • npm run knip — passed
  • npm run docs:check — passed
  • npm run metadata:check — passed
  • npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts tests/boundary/cli/analysisViewCli.test.ts tests/boundary/mcp/analysisViewMcp.test.ts tests/boundary/mcp/largeToolResult.test.ts tests/boundary/mcp/fridaMcp.test.ts tests/boundary/mcp/workflowSnapshotReplayMcp.test.ts tests/boundary/process/mcpVerifierResults.test.ts — 8 files, 111 tests passed
  • Real Kuna 1.988 stdio MCP and CLI workflow — results in the table above
  • After the repeated-link fix, npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts — 2 files, 13 tests passed. The real workflow and scale runs predate that small counting change; their records had no repeated links.
  • After merging main at 93b8b82db, which includes feat(native): deliver batch decompile as a retained summary #8631's changes to shared delivery code: npm run check:fast, format:check, knip, docs:check and metadata:check passed, and npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts tests/boundary/mcp/batchDecompileViewsMcp.test.ts tests/boundary/mcp/largeToolResult.test.ts tests/boundary/cli/analysisViewCli.test.ts tests/boundary/process/mcpVerifierResults.test.ts passed (6 files, 80 tests).
  • After the advertised-schema test change (7e788be) and merging main at 2ac6d3efe, npx tsc --noEmit and npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts passed.
  • After merging main at aba56df68, npx tsc --noEmit and npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts tests/boundary/cli/analysisViewCli.test.ts passed.
  • After merging main at 0e6943c9d, npx tsc --noEmit and the same 2 focused files passed (13 tests). Every required CI check had passed on 6dc651e88 before this merge.
  • After merging main at f95bdbce7, npx tsc --noEmit and npm run test:focused -- tests/boundary/mcp/evidenceLinkInspectionMcp.test.ts tests/boundary/mcp/evidenceRecordSelectionMcp.test.ts — 2 files, 13 tests passed. Every required CI check had passed on 9a62a45c3 before this merge.
  • npm run test:focused -- tests/acceptance/applications/bundleResourceRootCliMcp.test.ts tests/acceptance/tools/inspectorFileUrlIdentity.test.ts tests/acceptance/tools/reconstructionScopedUnknowns.test.ts — 3 files, 5 tests passed

Compatibility, safety, and release

  • Breaking changes or migration steps: none intended. A client that strictly validates the summary with an older copy of the schema would reject the two new fields; this repository's consumers parse the current schema.
  • Real Hopper/Ghidra, browser, or OS coverage: real Kuna 1.988 on macOS arm64 only.
  • Package or release metadata impact: none beyond the generated catalog.
  • Security, privacy, process, or containment review: reads only retained Evidence or the caller's bundle, under the existing bundle reader and its memory budget. Rows carry only Evidence IDs.

Author readiness

Human owner: @kaoru0822-kitauji

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

…records

An agent could only learn that a retained record's source was absent by reading
complete records or by having a downstream analysis refuse. The summary said
nothing about links, and complete-record retention was easy to mistake for
source closure.

- Add detail: links to get_evidence_bundle and CLI inspect-evidence-bundle:
  paged link rows with retained or missing status and no record payloads, for
  selected records against the whole session or file.
- Add retained-closure traversal, the walk composed workflows require, reporting
  complete or incomplete. Direct reads make no closure claim.
- Count direct and missing direct links in the summary.

Fixes #8454

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaoru0822-kitauji kaoru0822-kitauji added enhancement New feature or request track:feature Feature/enhancement automation track labels Nov 2, 2026
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T08:55:32.319602Z 017cda8 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

kaoru0822-kitauji and others added 5 commits November 2, 2026 06:09
The Evidence schema does not deduplicate evidence_links, so the summary counted
a repeated link twice while link rows and counts reported it once. Both now read
one helper for a record's distinct direct links.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] No blocking issues were found; the PR appears safe to merge.

Summary

Adds detail: "links" to get_evidence_bundle and inspect-evidence-bundle so callers can find absent source records without transferring record payloads.

  • Evidence summaries count each matching record's direct links.
  • Callers can inspect Evidence links without loading record payloads.
  • Complete-bundle tests still check the same records.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  MCP["MCP session snapshot"] --> Owner["inspectEvidenceBundle"]
  CLI["CLI authenticated bundle file"] --> Owner
  Owner --> Select["Select starting records"]
  Select --> Direct["Read direct links"]
  Select --> Closure["Follow retained links when requested"]
  Direct --> Membership["Check against every retained record"]
  Closure --> Membership
  Membership --> Counts["Count all inspected links"]
  Counts --> Page["Filter and page link rows"]
  Page --> Result["Return IDs and membership without record payloads"]
Loading

Reviews (3) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

kaoru0822-kitauji and others added 3 commits November 2, 2026 08:31
The test never called tools/list, so neither the shared validator nor the MCP
client checked its replies against what each profile advertises. A schema that
rejected the links result could have passed. Fetch the advertised definition and
validate every request and reply against it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kaoru0822-kitauji
kaoru0822-kitauji merged commit 710fad8 into main Nov 2, 2026
53 of 55 checks passed
@kaoru0822-kitauji
kaoru0822-kitauji deleted the feat/8454-evidence-link-inspection branch November 2, 2026 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request track:feature Feature/enhancement automation track

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Design] Discover missing Evidence links without full record transfer

1 participant