fix(test): bypass pnpm dependency checks in mutations - #1326
Conversation
APPROVEExact head: Negative-controlled the fix: yes. Misspelling Did 1. Central probeRAN (not read):
Independent of the author's summary: pnpm 11.1.2 (
2. Scope of the environment change
3. Regression controlRAN
Did not sample a third fixture family. PTC is a different family (script-field Independent census of 4. Claims
CI at
|
| Workflow | Present by name | Conclusion |
|---|---|---|
| Mutation reproof | PRESENT | success |
| CI | PRESENT | UNMEASURED (in_progress) |
| Windows | PRESENT | UNMEASURED (queued) |
| Code Quality: PR #1326 | PRESENT | success |
| Docs | correctly absent | path filter is website/**, docs/**, spec/**, SPEC.md, assets/**, .github/workflows/docs.yml; this PR touches only scripts/mutation-proof.mjs and scripts/mutation-proof.selftest.mjs |
| Installer | correctly absent | path filter is install.sh, install/**, .github/workflows/installer.yml |
Do not treat CI or Windows as green.
Could not check
- Full 342-fixture sweep (cost; sampled the broken fixture and one other family).
- CI / Windows conclusions at this sha (still queued / in progress).
- Live lockfile rewrite without
CI(would dirtypnpm-lock.yaml; body claim is consistent with pnpm 11.1.2case "install": install2()).
Ran vs read
Ran: pnpm install --frozen-lockfile; self-test; self-test with misspelled env; manager-runtime-deps mutation-proof with misspelled env; same after restore; package-test-contract mutation-proof; pnpm 11.1.2 source for the env mapping; issue-ref scan of the PR body.
Read: PR body, the two-file diff, fixture JSON, docs.yml / installer.yml path filters, existing empty-frontmatter changeset example.
Worktree source is clean (git status --porcelain empty). Temporary misspell of the env key was restored with git checkout HEAD -- scripts/mutation-proof.mjs before the positive control. No leftover mutation-proof processes.
mutation-proof consumed each run's output for counting and filtering (`progressCount`, the `expectRed` filter, `completionMarker`) and never printed it. A WRONG-RED therefore reported that the expected string was absent and never what was there instead. That is what made #1325 cost a day. `manager-runtime-deps` sat red on main, on every PR selecting it, and on the 0.46.0 release PR, with all four cells at `0 marks (baseline 54)` and no way to tell from the log why. The cause was a pnpm frozen-lockfile abort printing to the child's output, which the runner had captured (`output = stdout + stderr`) and then discarded. Non-KILL verdicts now echo that transcript, bounded to the last 20 lines, with a distinct line when the run produced nothing at all. A KILL prints none: its verdict already names the assertion that reddened. Controlled both ways rather than assumed. A deliberately bogus `expectRed` produces WRONG-RED and prints the suite's own output beneath it, ending in the `✗ FAIL:` line and `SUITE COMPLETE: 53 passed, 1 failed`; the four real cells come back KILLED and print nothing extra. Scope note: this deliberately does NOT fix the launcher defect behind #1325. That is #1326's, which sets `pnpm_config_verify_deps_before_run=false` in the spawned children and carries a negative control plus a self-test. I had a narrower fixture-level fix and dropped it: two mechanisms for one defect leaves the next reader unsure which is load-bearing, and it would have made `manager-runtime-deps` the only fixture invoking its suite differently from the other 341.
1 participant
Summary
mutation-proofCause
pnpm 11 defaults
verifyDepsBeforeRuntoinstallforpnpm runandpnpm exec. A mutation that changes dependency metadata intentionally desynchronizespackage.jsonfrompnpm-lock.yaml. Under CI's frozen lockfile, pnpm aborts before the configured suite starts, so mutation-proof sees zero progress marks instead of the guard's verdict.Without
CIset, the same launcher silently rewritespnpm-lock.yamlbefore running the suite. That side effect hid the failure locally and is recorded here as a tooling footgun. This change prevents it only inside mutation-proof child runs.There are 9 mutation cells across 5 fixtures that target a
package.json. Four manager-runtime-deps cells are demonstrably broken by dependency-metadata desynchronization. The remaining five are latent exposure, not known failures, because script-only manifest mutations can leave the lockfile consistent.Validation
node scripts/mutation-proof.selftest.mjs: 33 checks passedCI=1 node scripts/mutation-proof.mjs --config bin/smoke/mutations/manager-runtime-deps.json: baseline 54 marks, all 4 mutations KILLED on their named dependency checksCI=1 node scripts/mutation-proof.mjs --config bin/smoke/mutations/package-test-contract.json: baseline 79 marks, all 4 mutations KILLED; the comparison cells remained at 76 and 77 marksCloses #1325