perf: spill large page indexes into the db meta directory

[corylanou]

Stacked on #1479 (base branch fix/1477-serialize-db-maintenance); pairs with superfly/ltx#97 (tracked in superfly/ltx#96). Part of #1477.

Summary

• Enables the ltx encoder's page-index spill for snapshots and compactions, with DB.MetaPath() as the spill directory — it exists in every deployment, including the hardened FROM scratch image, which has no /tmp.
• Every ltx.NewCompactor site (compaction, restore, VFS hydration) and the snapshot encoder now defer Cleanup(), so a cancelled or failed operation cannot leave a spill file behind, and an abandoned encoder cannot later be closed into a checksum-valid file with an empty index (ltx#97 makes Cleanup terminal for that reason).
• With ltx#97's streaming decoder, compaction inputs no longer materialize a page-index map at all.
• go.mod pins ltx to the head of perf: stream the decoder page index and spill the encoder index to disk superfly/ltx#97 for evaluation. Replace with the tagged ltx release before merge (same as fix(store): serialize snapshots and compactions per database #1479's pin).

Evidence

litestream-soak snapshot-compaction-overlap rig (corylanou/litestream-soak#196): the snapshot's replica stream is held at 95% so its page index stays resident while the L1 compaction runs; heap growth = peak HeapInuse − post-GC baseline, GOGC=25, peak heap profile per phase. All rows below include per-database serialization (#1479); the ltx column is what changes.

build	fixture	snapshot	L1 compaction	overlap
#1479 + ltx v0.5.2	4 GiB / 1,051,216 pages	173.1 MB	300.7 MB	286.5 MB
#1479 + ltx#95 (as pinned there)	4 GiB	68.8 MB	203.4 MB	199.6 MB
this PR + ltx#97	4 GiB	70.4 MB	73.7 MB	73.1 MB
this PR + ltx#97	8 GiB / 2,102,433 pages	73.1 MB	72.9 MB	73.9 MB

Going from 4 to 8 GiB no longer moves any phase: ~32 MB is the fixed S3 multipart upload buffers, the rest is the ≤24 MiB of in-memory index chunks before the spill engages. Peak profile of the 8 GiB compaction phase is s3/manager.(*maxSlicePool).newSlice 32 MB + ltx.(*pageIndex).append 22.8 MB; DecodePageIndex (84 MB at 4 GiB before) is gone. For comparison, litestream main with ltx v0.5.2 measured 470.6 MB for the 4 GiB overlap.

At the reporter's 34.2M pages this leaves maintenance memory at roughly the S3 buffer allowance plus a few tens of MB, versus ≈15 GB on main and ≈6.5 GB with #1479 + ltx#95.

Follow-up

• Hydrator.ApplyLTX (vfs.go) reads pages and returns without dec.Close(), so it never verifies the index/trailer/file checksum. Pre-existing; it should close the decoder with retention off. Separate PR.

Test plan

• go test ./ root suite green against ltx#97; go vet, pre-commit (goimports, vet, staticcheck)
• Rig evidence above at 4 GiB and 8 GiB, profiles saved
• ltx side reviewed in four adversarial Codex passes (see ltx#97)

[github-actions]

PR Build Metrics

✅ All clear — no issues detected

Check	Status	Summary
Binary size	✅	37.19 MB (+16.0 KB / +0.04%)
Dependencies	ℹ️	1 added, 1 removed
Vulnerabilities	✅	None detected
Go toolchain	✅	1.25.14 (latest)
Module graph	✅	1230 edges (0)

Binary Size

	Size	Change
Base (10d628d)	37.17 MB
PR (07272a1)	37.19 MB	+16.0 KB (+0.04%)

Dependency Changes

Added:

• github.com/superfly/ltx v0.5.3-0.20260828134549-493c5d2d9e9f

Removed:

• github.com/superfly/ltx v0.5.3-0.20260827162011-d457a1ab7844

govulncheck Output

=== Symbol Results ===

No vulnerabilities found.

Your code is affected by 0 vulnerabilities.
This scan also found 1 vulnerability in packages you import and 3
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.

Build Info

Metric	Value
Build time	50s
Go version	go1.25.14
Commit	07272a1

History (1 previous)

Commit	Updated	Status	Summary
e2ea295	2026-08-28 13:48 UTC	✅	37.19 MB (+16.0 KB / +0.04%)

---

_{🤖 Updated on each push.}