test(manager,connector-core): make the mutation reproof gate grade what it claims - #1329
Conversation
A `completionMarker` says "a run of mine that did not finish is not evidence I want counted". These three carried the suite's cell count inside that string, which is a second assertion the suite already makes for itself: each prints `SUITE COMPLETE: <ran> cells` and separately refuses a run whose `ran` is not its EXPECTED_CELLS. The duplicate rotted. A cell was added to the connection-status suite on 2026-08-30 and the fixture still asked for 15, so a finished 18-cell run printed a line the marker could never match and all seven mutants graded INCONCLUSIVE. Nothing failed and the fixture kept reporting; the mutation reproof gate simply stopped grading that file. The other two markers are correct today and rot the same way on the next added cell. All seven connection-status mutants now grade KILLED, transport-liveness 18, transport-liveness-broker 3.
…sal fails Three suites could reach their last cell and still print nothing, so a run that had already reddened the right cell looked like a run that never happened. events-grant-acl had no catch at all. Its later cells feed on earlier ones: a rule that stops refusing lets the spawn through, and the section that parses the refusal then mints on an empty profile and dies inside `mintCreds`, past the finally and before the summary. It now records the throw as a failure and falls through to its own verdict. user-spawn printed its summary only on the success path. The same cascade shape applies: the refusal text is missing, the flags parsed out of it are empty, and the real ledger writer refuses them. The catch prints the summary too. boot-self-heal-gate hung. Two cells expect a successor start to REFUSE and neither stopped one that started anyway, so a live manager with timers and a broker connection held the event loop open and the process never exited. A hang is not a red, so the mutation that caused it graded as no evidence. E1, E9, E11, R1, R2 and the assume-dead mutant were the six that could not be graded. All six now grade KILLED, along with every other mutant in those three fixtures.
The boot-wake-race fixture mutated the redelivery re-announce in connector-core, from a time when that suite's recovery ran through JetStream redelivery. It does not any more. The suite settles each scenario inside a 500ms window chosen so no redelivery can have happened, and its own header says the redelivery guard lives in `smoke:cross-path-dedup` and `smoke:claude-wake`. So the mutation changed nothing the suite observes, the suite passed with it applied, and it graded UNGRADABLE. The re-announce moves to a fixture on cross-path-dedup, which drives both deliveries by hand and reads MeshAgent from source. Two mutants, one per line of the guard, so each is the other's positive control. boot-wake-race keeps a mutation on what it does drive: activation reconciling one buffered wake, with nothing else in the window able to produce a nudge. The mention branch above it stays, so the focus-mention scenario is unaffected. All three grade KILLED.
d98428c to
dedf9a2
Compare
Retrospective verdict: BLOCKExact head reviewed: Merge Blocking findingThe changed-set mutation reproof selector does not select fixtures whose suite changed.
That matters directly here. Exact-head CI reported these six selected fixtures:
It omitted fixtures whose grading behavior this PR changes through suite-only edits, including:
The exact gate therefore stayed green without enforcing most of the PR's core harness repairs. Required repair: retain Evidence that the individual repairs are otherwise soundI built a clean scratch archive and ran these mutation proofs there:
Exact-head CI also showed all verdicts in its six selected fixtures KILLED, with 0 pre-red and 0 inconclusive. Baseline CI showed cross-path dedup 54/54, Claude boot-wake 12/12, events-grant all 41 expected cells, and boot-self-heal 12/12. PR #1316 overlapThe PR head itself does not contain #1316's patch and still lists PR #1329 did not cause that defect, but it did not repair the merged artifact either. Run disclosureI did not run |
|
BLOCK at exact head Merge Finding: Independent dry-run at merge Selected:
Missed:
Exact-head CI job Required repair: keep #1316 overlap: this PR head does not contain #1316's patch. The merged artifact inherits #1316 as parent 1. At Run/read: no |
Acceptance traceability supplementVerdict remains BLOCK at This maps every changed output to an observed check rather than relying on aggregate CI status.
The PR-head corpus has eight suite-only misses:
The merge artifact has nine because parent 1 adds Observed conclusion: the individual fixture repairs are mostly better and discriminating. The public changed-set gate still returns a false all-clear for suite-only changes, and one changed catch arm lacks acceptance evidence because that omitted fixture is live-only. This directly confirms the BLOCK rather than inferring it from source inspection. |
1 participant
Fourteen mutants across five fixtures could not be graded at all: thirteen graded INCONCLUSIVE and quietly stopped grading, one graded UNGRADABLE. None of it was caused by the code under test: the same verdicts reproduce on a built checkout of
main. A sixth fixture,lease-renew, had the same shape and landed separately in #1327, so it is not in this change.Three separate defects, all in the tests.
Completion markers carried a cell count
connection-status,transport-livenessandtransport-liveness-brokereach declared a marker of the formSUITE COMPLETE: N cells. Each suite printsSUITE COMPLETE: <ran> cellsand separately refuses a run whoseranis not itsEXPECTED_CELLS, so the count inside the marker duplicates a check the suite already makes, and it is the half that rots.It rotted. A cell landed in the connection-status suite on 2026-08-30 and the fixture still asked for 15, so a finished 18-cell run printed a line the marker could never match and all seven mutants graded INCONCLUSIVE. The markers are now count-free and the suites' own completeness check stands alone.
Three suites could end without printing a verdict
events-grant-aclhad no catch. Its later cells feed on earlier ones, so a rule that stops refusing lets a spawn through and the section that parses the refusal then mints on an empty profile and dies insidemintCreds, past the finally and before the summary.user-spawnprinted its summary only on the success path, with the same cascade shape behind it.boot-self-heal-gatehung. Two cells expect a successor start to refuse and neither stopped one that started anyway, so a live manager held the event loop open and the process never exited.Each now reports what it proved. A hang and a silent death both grade as no evidence, which is the opposite of what a cell that just went red is for.
One guard was not graded by any suite that runs it
boot-wake-racemutated the durable re-announce in connector-core, from a time when its recovery ran through JetStream redelivery. The suite settles each scenario in a 500ms window chosen so no redelivery can have happened, and its own header says that guard moved tosmoke:cross-path-dedup. The mutation moves there, split into two mutants so each is the other's positive control, and boot-wake-race keeps a mutation on the activation reconcile it does drive.Verification
Every affected fixture re-proved on a clean tree, plus the toolchain gates.
pnpm typecheck,pnpm smoke:mutation-fixturesandpnpm smoke:mutation-proofall pass.No shipped code changes, so no changeset.