Skip to content

fix(replica): upload only missing L0 gap files - #1462

Draft
corylanou wants to merge 1 commit into
issue-1151-failed-upload-breaks-compactionfrom
issue-1457-gap-heal-upload-only-missing
Draft

fix(replica): upload only missing L0 gap files#1462
corylanou wants to merge 1 commit into
issue-1151-failed-upload-breaks-compactionfrom
issue-1457-gap-heal-upload-only-missing

Conversation

@corylanou

Copy link
Copy Markdown
Collaborator

Summary

During L0 gap healing, upload only TXIDs absent from the remote L0 listing. Existing files above the gap are left untouched while the replica position still catches up.

This PR stacks on #1155 and must remain based on issue-1151-failed-upload-breaks-compaction until that PR merges.

Problem

The #1155 recovery path recalculates the replica position just before an interior L0 gap. The sync loop then uploads the entire suffix even though calcPos already listed the files that exist above the gap.

The regression test demonstrated the behavior before this fix: deleting TXID 3 from a remote 1-4 listing caused WriteLTXFile calls for both TXIDs 3 and 4. Only TXID 3 was missing.

Solution

Return the sorted remote L0 listing from calcPos and retain it for the catch-up sync. The upload loop advances past TXIDs covered by that listing and calls WriteLTXFile only for missing TXIDs. The listing remains available across upload-limited sync batches and is cleared after catch-up or an error.

Behavior Change

Gap-heal action Before After
Missing L0 TXID Uploaded Uploaded
Existing L0 TXID above gap Re-uploaded Skipped
Final replica position Catches up Catches up

Scope

In scope:

  • Reuse the remote L0 listing during gap heal
  • Preserve existing gap recovery and compaction semantics
  • Assert that only the missing L0 TXID is written

Not in scope:

  • Dual-writer lineage detection
  • Conditional backend writes

Test Plan

  • GOTOOLCHAIN=go1.25.13 go test -race -run "^(TestReplica_InvalidatePos_HealsL0Gap|TestDB_Compact/L0GapHealsViaReplica)$" -count=1 .
  • GOTOOLCHAIN=go1.25.13 go test -race ./...
  • GOTOOLCHAIN=go1.25.13 pre-commit run --all-files
  • TestLTXBehavior: low-volume, high-volume, and burst-volume profiles passed all applicable assertions and restore integrity checks
  • GOTOOLCHAIN=go1.25.13 go test -tags "integration,soak" -run "^TestLTXBehavior_NoExcessiveSnapshots$" -short -v -timeout 5m ./tests/integration/

Related

Gap healing rewinds to before the first missing remote L0 TXID, but
sync previously rewrote every later file. Retain the calcPos listing
during sync and skip TXIDs it already covers so only missing files are
uploaded.

Fixes #1457
@github-actions

Copy link
Copy Markdown

PR Build Metrics

⚠️ Attention needed — Go toolchain outdated (1.25.13 → 1.25.14)

Check Status Summary
Binary size 37.17 MB (+4.0 KB / +0.01%)
Dependencies No changes
Vulnerabilities None detected
Go toolchain ⚠️ 1.25.13 → 1.25.14 available
Module graph 1230 edges (0)

Binary Size

Size Change
Base (ffe640d) 37.17 MB
PR (566c142) 37.17 MB +4.0 KB (+0.01%)

Dependency Changes

No dependency changes.

govulncheck Output

=== Symbol Results ===

No vulnerabilities found.

Your code is affected by 0 vulnerabilities.
This scan also found 1 vulnerability in packages you import and 3
vulnerabilities in modules you require, but your code doesn't appear to call
these vulnerabilities.
Use '-show verbose' for more details.

Build Info

Metric Value
Build time 43s
Go version go1.25.13
Commit 566c142

🤖 Updated on each push.

@github-actions github-actions Bot added the metrics: go-update Go toolchain has a newer patch release label Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

metrics: go-update Go toolchain has a newer patch release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant