Immutable/volatile split of LeiosDB + garbage collection - #2223
Draft
geo2a wants to merge 3 commits into
Draft
Conversation
geo2a
force-pushed
the
geo2a/leios-969-leiosdb-gc
branch
4 times, most recently
from
August 20, 2026 06:41
e3544e6 to
dabc62d
Compare
- Add a column to the `ebs` SQLite table that distinguishes EBs as immutable.
- Mark EBs as immutable once their RB becomes immutable.
- Garbage Collect EBs and their transations if they are volatile and are older that
the immutable tip.
- Add trace events for Prometheus gauges:
* `TraceLeiosDbCopiedToImmutable` --- Rows moved into the immutable partition
by 'LeiosDbHandle.leiosDbCopyToImmutable'
* `TraceLeiosDbVolatileStats` --- volatile counts and file bytes, sampled on a
timer over a fresh read-only connection per tick, kept clear of startup.
* `TraceLeiosDbImmutableStats` --- the immutable counts maintained incrementally
in a one-row `immutableStats` table.
geo2a
force-pushed
the
geo2a/leios-969-leiosdb-gc
branch
from
August 20, 2026 12:25
dabc62d to
7c1058f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements input-output-hk/ouroboros-leios#969
cardano-nodePR: IntersectMBO/cardano-node#6656This PR splits the contents of LeiosDB into volatile and immutable partitions, and periodically marking EBs as immutable once their referencing RB becomes immutable and garbage collection orphaned volatile EBs.
An EB is marked as immutable via the
immutableflag column of theebstable. All transactions (rows of thetxstable) referenced by this EB via theebTxstable become immutable as well, but there are no explicit marker in those tables.The PR also adds observability of the volatile/immutable EB counts an the GC rates, which is wired into a Graphana dashboard which looks like this:
Benchmarking
The PR adds the
leios-gc-benchbenchmark, which works both synthetic data and with a realleios.db. I've generated some input data by running the devnet overnight to test the performance:The duration of the first immutable marking and GC are an outlier, which Claude attributes to cold OS page cache. The outliers are clearly visible on the following scatter plot (x axis is log scale):
When running the benchmark with synthetic data, the outliers don't show up, probably because the sqlite database is written into before the start of the measurement:
Implementation details
One particular peculiarity that I had to iron out was the use of the
unsafeFFI calls to the SQLite bindings. The immutalisation or GC operations need to run in asafeFFI call so that they are run on a separate GHC RTS capability and do not block the node from doing other things. This needs further investigation, together with auditing other FFI calls and making a weighted decision on which mechanism to better for every one of them.