fix(vfs): stop wedging the level-1 tailer on a straddling first compaction - #1465
Open
yzxcj797 wants to merge 1 commit into
Open
fix(vfs): stop wedging the level-1 tailer on a straddling first compaction#1465yzxcj797 wants to merge 1 commit into
yzxcj797 wants to merge 1 commit into
Conversation
…ction rebuildIndex seeded maxTXID1 from the L0-derived pos.TXID when the index was built while L1 was still empty. That value is not an L1 boundary: the writer's first L0-to-L1 compaction emits a file whose TXID range straddles it (e.g. 1..0xb around a position of 6). The level-1 seek skips files by MinTXID, so the straddling file is never returned, and once the next L1 file appears (0xc..0x1d) the strict contiguity check fails forever: poll L1: non-contiguous ltx file: level=1, current=6, next=c-1d The follower freezes at TXID 6 while still serving stale reads, even though storage is fully contiguous. Leave maxTXID1 at zero when there are no L1 files, so the first L1 file is consumed contiguously from TXID 1 and the position tracks real L1 boundaries from then on. The vfs-tagged test build was broken on main (the test-file mocks predate the SetLogger addition to ReplicaClient); the mocks are updated so the tagged suite compiles again. Fixes benbjohnson#1460
Contributor
Author
|
@darkgnotic thanks for flagging that — I checked the write-mode path against #953 and I don't see an interaction:
So the effect is strictly within the read-side tailer that the issue is about. |
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1460.
Investigation (reproduced from the report's timeline)
rebuildIndexseeds the level-1 tailer position from an L0-derived value when the index is built while L1 is still empty:The writer's first L0→L1 compaction then emits a file whose range straddles that value (
1..0xbaround position 6). The level-1 seek filters byMinTXID, so the straddling file is invisible forever; when the next L1 file (0xc..0x1d) appears, the strict contiguity check fails on every poll and the follower freezes at TXID 6 while storage remains fully contiguous:Fix
maxTXID1stays zero when there are no L1 files, so the first L1 file — which starts at TXID 1 regardless of how far L0 has advanced — is consumed contiguously, and the position tracks real L1 boundaries from then on.Scope
rebuildIndex(plus tests). It does not change theMinTXID-based seek semantics of any storage backend, nor thepollLevelcontiguity rules — with a correctly-zero seed neither can encounter a straddle.SetLoggeraddition toReplicaClient); the three mocks get a no-opSetLoggerso the tagged suite builds again. Pre-existing failures unrelated to this change (e.g.TestReplica_SyncOnceLimitsLTXFiles,TestDB_SyncChunksWALAtCommitBoundary) reproduce identically with and without this fix.Tests
Updated
TestVFSFile_OpenSeedsLevel1PositionFromPos(which pinned the old seeding) toTestVFSFile_OpenKeepsLevel1PositionAtZeroWithoutL1Files, and addedTestVFSFile_PollConsumesStraddlingL1Compaction, which replays the report's exact timeline: open with L0 at 6 and L1 empty, then present L11..0xband0xc..0x1d, then poll.Differential: reverting only the seeding (tests kept) fails with exactly the report's wedge error —
poll L1: non-contiguous ltx file: level=1, current=0000000000000006, next=000000000000000c-000000000000001d.