Skip to content

feat(web-api-discovery): name each post-processing stage and heartbeat the AI pass (#492) - #506

Merged
MarkMichaelis merged 3 commits into
mainfrom
feat/492-post-process-progress
Sep 14, 2026
Merged

feat(web-api-discovery): name each post-processing stage and heartbeat the AI pass (#492)#506
MarkMichaelis merged 3 commits into
mainfrom
feat/492-post-process-progress

Conversation

@MarkMichaelis

@MarkMichaelis MarkMichaelis commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

What

After a recording stops, HAR post-processing ran for minutes and printed nothing -- ~95% of that time in the spawned claude -p catalogue pass. The operator could not tell a working run from a hang.

  • Each post-processing stage is now named as it starts: scrubbing (with the size), verifying the scrub, building the digest, cataloguing.
  • The AI catalogue child now runs under a small wrapper that prints an elapsed-time line every 30 s naming the child's pid, and a closing line when it ends. The child's own output still goes straight to the terminal; its exit status passes through (a signal-killed child reports 128 + signal number, so it stays distinguishable from the CLI exiting 1).

Heartbeat rather than streaming, as the issue recommends; streaming stays a separate question. Post-processing stays synchronous -- the wrapper is the process that can keep a timer while the recorder blocks on the child.

Evidence

Same fixture, before -> after: nothing printed -> three stage lines. Heartbeat at its real interval against a 65 s silent child:

[  0.2s] child: working (silent under -p, like the real catalogue pass)
[ 30.1s] capture-har: cataloguing ... 30s elapsed (pid 55064)
[ 60.1s] capture-har: cataloguing ... 1m00s elapsed (pid 55064)
[ 65.2s] capture-har: cataloguing finished after 1m05s
exit: 0

Tests

  • New capture: post-processing prints nothing for minutes, so a working run is indistinguishable from a hang #492 section: stages announced before they run and in order; digest announced before it is written; the real pipeline launches the AI child through the wrapper (stdio inherited, cwd = output path, announced first); elapsed-time format; heartbeat names the child pid; heartbeat stops with the child; exit status passes through; signal deaths map to 128 + signal; a child that cannot start is reported, not waited on; the script door passes status through.
  • Red first; ablations: removing the timer cleanup and bypassing the wrapper each fail their test.
  • Local CI (GitHub Actions is billing-blocked): capture node suites 82 + 16 + 29 + 18 + 14 + store + 13, all green; capture Pester wrappers + node-test-coverage 153 passed, 0 failed.

Independent review

Reviewer model: Claude Sonnet 5 (author: Opus 5), two rounds.

  • Round 1: no Critical; 2 Important accepted and fixed (real launch path untested -> injectable launcher + wiring test; signal deaths collapsed to exit 1 -> 128 + signum), 1 Minor accepted (spurious "could not start" after a real exit), 1 Minor rejected (a few hundred characters less command-line headroom; same single process hop).
  • Round 2: fixes confirmed; no new Critical or Important findings.

Closes #492
Refs #491

🤖 Generated with Claude Code

https://claude.ai/code/session_01CUt6s3N3iqMMEtvd9AoAnU

MarkMichaelis and others added 2 commits September 13, 2026 20:28
…ng (#492)

Stages must be announced BEFORE they run and in pipeline order; the AI
catalogue child must run under a heartbeat wrapper that names its elapsed
time and pid, stops when the child does, passes the exit status through,
and reports a child that cannot start instead of waiting on it.

Refs #492

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CUt6s3N3iqMMEtvd9AoAnU
…t the AI pass (#492)

After a recording stops, post-processing ran for minutes printing nothing,
~95% of it the spawned `claude -p` catalogue pass -- indistinguishable from
a hang. Each stage (scrub, with its size; verify; digest; catalogue) is now
named as it starts, and the catalogue child runs under run-with-heartbeat.js,
which prints an elapsed-time line every 30 s naming the child's pid and a
closing line when it ends. postProcess stays synchronous: the wrapper is the
process that can keep a timer while spawnSync blocks.

Closes #492

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CUt6s3N3iqMMEtvd9AoAnU
@MarkMichaelis

Copy link
Copy Markdown
Contributor Author

Evidence -- #492: post-processing says what it is doing

Change type: CLI behaviour (bug fix: silence indistinguishable from a hang).

1. Stage labels -- same fixture, before and after

Synthetic 400-entry capture (166.5 KB), run through the real postProcess
(real scrub, real leak gate). Timestamps are seconds since the call.

Before (origin/main):

(postProcess returned: errors=0, verified=true, catalogue delegated to agent)

Nothing at all is printed while the stages run.

After (this branch):

[0.0s] capture-har: scrubbing 166.5 KB ...
[1.2s] capture-har: verifying the scrub ...
[1.4s] capture-har: building the digest ...
(postProcess returned: errors=0, verified=true, catalogue delegated to agent)

Each stage is named as it starts. The catalogue label is not shown here
because this run delegated cataloguing to the calling agent -- no AI child
was spawned, which is correct.

2. The heartbeat, at its real 30-second interval

The wrapper the AI catalogue child now runs under, driving a stand-in child
that prints one line and then stays silent for 65 s (as claude -p does).
Left column is wall-clock seconds.

[  0.2s] child: working (silent under -p, like the real catalogue pass)
[ 30.1s] capture-har: cataloguing ... 30s elapsed (pid 55064)
[ 60.1s] capture-har: cataloguing ... 1m00s elapsed (pid 55064)
[ 65.2s] capture-har: cataloguing finished after 1m05s
exit: 0

The child's own output still reaches the terminal directly, the heartbeat
names the child's pid, and the wait ends with a closing line.

Review

  • A. Does the artifact confirm the intent? Yes -- every stage is
    announced before it runs, and the silent AI stage now shows an
    elapsed-time heartbeat with the child's pid, matching the issue's
    proposed output.
  • B. Any new problem? None seen: no stdout output was added (labels are on
    stderr), results and exit status are unchanged, the heartbeat stops
    when the child ends.

- The claude-cli branch now takes an injectable launcher and runner
  context, so a test proves post-processing really launches the AI child
  through the heartbeat wrapper -- previously only the argv helper was
  tested, and reverting the launch line left the suite green.
- A child killed by a signal reports 128 + the signal number instead of a
  plain 1, so "catalogue: claude exited N" still distinguishes a killed
  pass from the CLI failing.
- A late error event after a real exit no longer prints "could not start".

Reviewer: Claude Sonnet 5 (different model from the author).

Refs #492

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CUt6s3N3iqMMEtvd9AoAnU
@MarkMichaelis
MarkMichaelis merged commit 71ebe8b into main Sep 14, 2026
2 checks passed
@MarkMichaelis
MarkMichaelis deleted the feat/492-post-process-progress branch September 14, 2026 03:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

capture: post-processing prints nothing for minutes, so a working run is indistinguishable from a hang

1 participant