feat(tailcall): model march's Pass 3 tail-call enforcement - #22
Conversation
Closes a live FALSE ACCEPT class: march ERRORs on truly unbounded non-tail recursion (`enforce_tail_calls_in_decls`, typecheck.ml:11367 → check_recursion_safety, :10715), and nothing here modelled it. `loopy`, `go` and `nested_mod_flat` accepted at 0 where march exits 1; `mutual` skipped at 2. `MarchLean/TailCall.lean` ports the analysis 1:1 from march's own code — the extern-name subtraction, the scope-threaded shadowing discipline, and `is_structurally_smaller` verbatim (getting the last one too NARROW is what would manufacture false rejects, since march merely WARNS on structural recursion and allows it). It reads the raw JSON envelope rather than the decoded `Module`, the only pass here that does. `Elab` is lossy in exactly the four places Pass 3 is structural: it drops `fn.attrs` (so `@[no_warn_recursion]` — live in march's own stdlib — would be invisible), and collapses ECond/ELetQ/ELetFn to `Term.opaque_`, erasing the tail positions and binders the analysis is made of. Reading the envelope needs no change to Syntax/Elab, so no existing verdict can be perturbed by construction. Runs pre-gate, beside CapCheck, mirroring march's own ordering. It returns reject-or-silence and never licenses an accept. Seeing out-of-fragment modules is what lets `mutual` flip 2 → 1 without waiting for af6fece. Two findings that reading the source would not have produced: - march's Pass 3 does NOT fire inside a nested `mod`, despite having a DMod arm that looks load-bearing. Recursing there would have been a false reject. Confirmed three ways (no error, no structural warning, and nested type errors unreported); `nested_mod` pins it. - The runner must not use `pipefail`: march exits 1 on --emit-core-ast for a rejected file, which masked march-lean-check's own exit and turned every false accept into a spurious "ok" on the suite's first run. Verification. 22 probes in scripts/tailcall-probes/, in CI before the harness. Every `clean` probe was mutation-tested to confirm it bites, which caught three that passed for the wrong reason: `cond_tail` recursed structurally (so tail position was never tested), and the `shadow_*` trio guarded only the walk, leaving the call graph unguarded — hence the new `shadow_edge_*`, `lambda_edge` and `letfn_edge` probes. Three native_decide guards over real emitter envelopes make a regression a build failure. Corpus unchanged at MATCH 86 / MISMATCH 0 / SKIP 246 / PASS (it contains no such recursion, so it moves nothing). Broadest false-reject evidence: 124 march-accepted stdlib+examples files, 0 tail-call rejects.
…cause Two corrections to the nested-`mod` section, one factual and one substantive. The claim that a type error inside a nested `mod` also goes unreported was WRONG. It came from a measurement that piped march through `head` and read `$?` — which is `head`'s status, not march's. Nested `mod` bodies are typechecked normally. The Pass 3 findings are unaffected: those were measured with `grep -c` on the output, and both still hold. The real reason Pass 3 finds nothing inside a nested `mod` has since been traced upstream. `Desugar.qualify_module_refs` rewrites bare intra-module CALL SITES inside every nested `DMod` to `Prefix.name` and leaves the DECLARATION name bare, so Pass 3 searched a post-desugar body for a pre-desugar name. The `DMod` arm does run; it just never matches. That is a march bug, fixed upstream on `fix/tailcall-nested-mod-qualified-names`. Consequence recorded in both places: when this repo re-pins to a march carrying that fix, `nested_mod` flips from `clean` to `tailcall` and the pass must model the prefix. That probe going red is the intended signal.
|
Upstream fix for the nested- That PR makes march reject nested- Root cause, for the record: |
Closes a live FALSE ACCEPT class. march ERRORs on truly unbounded non-tail recursion (
enforce_tail_calls_in_decls,typecheck.ml:11367→check_recursion_safety,:10715) and nothing here modelled it.loopy—loopy(n + 1) + 1go— same shapemutual—walk/helperSCCnested_mod_flatfact—fact(n - 1) * n, structuralcleanprobesmutualflipped without waiting foraf6fece(still unmerged onclaude/mutual-recursion), because the pass runs pre-gate.Design
MarchLean/TailCall.leanports the analysis 1:1 from march's own code: the extern-name subtraction, the scope-threaded shadowing discipline, andis_structurally_smallerverbatim. Getting that last one too narrow is what manufactures false rejects — march merely WARNS on structural recursion and allows it, sofact(n - 1) * nmust keep passing.It reads the raw JSON envelope, not the decoded
Module— the only pass here that does.Elabis lossy in exactly the four places Pass 3 is structural: it dropsfn.attrs(so@[no_warn_recursion], live in march's own stdlib, would be invisible), and collapsesECond/ELetQ/ELetFntoTerm.opaque_, erasing the tail positions and binders the analysis is made of. Reading the envelope needs no change toSyntax.leanorElab.lean, so no existing verdict can be perturbed by construction.Placement: pre-gate, beside
CapCheck, mirroring march (its capability checks attypecheck.ml:11351precede Pass 3 at:11367).TailResulthas noskipcase — the pass returns reject-or-silence and never licenses an accept.Bail rule: an unrecognised expression or pattern
kindanywhere in a function body abandons that function. Bail beats error, since an unmodelled block sibling could have been anELetshadowing the recursive name.Two findings that came from probes, not from reading the source
march's Pass 3 does not fire inside a nested
mod.enforce_tail_calls_in_declshas aDModarm that recurses into inner decls and looks entirely load-bearing. It is dead in the--checkpath. Confirmed two ways: no error, and no structural-recursion warning either — so it is Pass 3 as a whole that finds nothing inside, not just its error path. Following the source here would have been a false reject.Root cause, traced upstream after this branch was written:
Desugar.qualify_module_refsrewrites bare intra-module call sites inside every nestedDModtoPrefix.name(EVar "boom"→EVar "Inner.boom") and leaves the declaration name bare. Pass 3 builtfn_namesfrom the bare declaration name, so it searched a post-desugar body for a pre-desugar name and matched nothing. TheDModarm does run — it just never matches.That is a march bug, now fixed upstream on
fix/tailcall-nested-mod-qualified-names. When this repo re-pins to a march carrying that fix,nested_modflips fromcleantotailcalland the pass must model the prefix. That probe going red is the intended signal — it is the whole reason march is modelled as it behaves rather than as its source reads.Correction to an earlier revision of this description: it also claimed a type error inside a nested
modgoes unreported. That was wrong — the measurement piped march throughheadand read$?, which ishead's status. Nestedmodbodies are typechecked normally. The Pass 3 findings usedgrep -con the output and are unaffected.The probe runner must not use
pipefail. march exits 1 on--emit-core-astfor a rejected file, which masksmarch-lean-check's own exit — turning every false accept in the suite into a spurious "ok". This bit on the suite's first run.Verification
Every
cleanprobe was mutation-tested — break the guard it protects, confirm it goes red. That caught three probes passing for the wrong reason:cond_tailrecursed asspin(n - 1), which is structurally smaller and therefore allowed whether or not the arm body is tail position. It tested nothing. Rewritten tospin(dec(n)).shadow_*trio guarded only the walk, leaving the call graph unguarded → addedshadow_edge_*.callstreatingELam/ELetFnbodies as new scopes → addedlambda_edge,letfn_edge.The full guard → probe table is in §8 of the design doc.
scripts/tailcall-probes.sh), wired into CI before the harness so a false reject fails fast.native_decideguards over real emitter envelopes — a regression breaks the build, no march binary required.MATCH 86 / MISMATCH 0 / SKIP 246 (145 + 101) / KNOWN_LIMITATION 2 / PASS, identical to baseline. It moved not one file, exactly as expected: the corpus contains no unbounded non-tail recursion. It proves no regression and nothing more.stdlib/): every march-accepted file under march'sstdlib/andexamples/— 124 files, 0 tail-call rejects. Since march accepts them all, any reject would be a false reject by definition. Exercises real@[no_warn_recursion]uses inhamt.marchanddataframe.march.dataframe.march(3520 lines, 4 MB envelope) checks in 0.12 s.Design doc:
specs/plans/2026-08-09-tail-call-enforcement-design.md.