fix(replica): upload only missing L0 gap files - #1462
Draft
corylanou wants to merge 1 commit into
Draft
Conversation
Gap healing rewinds to before the first missing remote L0 TXID, but sync previously rewrote every later file. Retain the calcPos listing during sync and skip TXIDs it already covers so only missing files are uploaded. Fixes #1457
PR Build Metrics
Binary Size
Dependency ChangesNo dependency changes. govulncheck OutputBuild Info
🤖 Updated on each push. |
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
During L0 gap healing, upload only TXIDs absent from the remote L0 listing. Existing files above the gap are left untouched while the replica position still catches up.
This PR stacks on #1155 and must remain based on issue-1151-failed-upload-breaks-compaction until that PR merges.
Problem
The #1155 recovery path recalculates the replica position just before an interior L0 gap. The sync loop then uploads the entire suffix even though calcPos already listed the files that exist above the gap.
The regression test demonstrated the behavior before this fix: deleting TXID 3 from a remote 1-4 listing caused WriteLTXFile calls for both TXIDs 3 and 4. Only TXID 3 was missing.
Solution
Return the sorted remote L0 listing from calcPos and retain it for the catch-up sync. The upload loop advances past TXIDs covered by that listing and calls WriteLTXFile only for missing TXIDs. The listing remains available across upload-limited sync batches and is cleared after catch-up or an error.
Behavior Change
Scope
In scope:
Not in scope:
Test Plan
Related