experiments(test-fill): add local t8n cache with SQLite + pickle + zlib

[spencer-tb]

🗒️ Description

Experimental: Adds an optional disk cache for t8n results that persists across fill runs.

This PR adds a content addressed SQLite cache that stores t8n outputs keyed by:

• Spec hash: SHA-256 of the EELS source code (per fork, including shared modules). Automatically invalidates when spec code changes.
• Content hash: SHA-256 of the t8n call inputs (fork, chain_id, reward, alloc, env, txs, blob_params, state_test).
  • Uses the pre-computed state_root from LazyAlloc for zero cost alloc hashing on subsequent blocks.

Entries are pickle serialized and zlib-compressed (level 1). SQLite WAL mode handles concurrent xdist workers. Transition forks hash both source and target fork directories.

Benchmarks

--fork Amsterdam -n 8 (35,716 tests), cache size 402MB:

Run	Time	vs No Cache
No cache	2:30	baseline
Cold	2:30	~1x
Warm	1:15	~2x faster

--until Amsterdam -n 8 (132,090 tests), cache size 2.5GB:

Run	Time	vs No Cache
No cache	10:53	baseline
Cold	12:37	~0.9x (slightly slower)
Warm	5:59	~1.8x faster

--generate-all-formats --fork Amsterdam -n 8 (50,035 tests), cache size 12GB:

Run	Time	vs No Cache
No cache	33:53	baseline
Cold	24:01	~1.4x faster
Warm	5:40	~6x faster

--generate-all-formats --until Amsterdam -n 8 (175,569 tests), cache size 38GB:

Run	Time	vs No Cache (~1:54)
No cache	<1:54:15	baseline
Cold	1:54:15	~1x
Warm	45:02	~2.5x faster
Warm (-n 12)	34:02	~3.4x faster

Usage

# Enable cache with the flag
uv run fill --fork Amsterdam --cache

# Custom DB path
uv run fill --fork Amsterdam --cache --cache-dir /path/to/cache.db

# Clear cache
rm .t8n-cache.db*

Open Questions

• Disk usage: 38GB for all forks + all formats (zlib compressed), quite large!
• Needs strategy for CI: per fork caching?
  • S3 FUSE mount? Download into runners? Only for forks <Osaka?
• Spilt cache by fork, given the cache is faster?
• Useful for local or CI only?

🔗 Related Issues or PRs

N/A.

✅ Checklist

• All: Ran fast static checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  just static

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered updating the online docs in the ./docs/ directory.
• All: Set appropriate labels for the changes (only maintainers can apply labels).

Cute Animal Picture

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.24%. Comparing base (cc21e89) to head (3f61e17).

Additional details and impacted files

@@               Coverage Diff                @@
##           forks/amsterdam    #2636   +/-   ##
================================================
  Coverage            86.24%   86.24%           
================================================
  Files                  599      599           
  Lines                36984    36984           
  Branches              3795     3795           
================================================
  Hits                 31895    31895           
  Misses                4525     4525           
  Partials               564      564           

Flag	Coverage Δ
unittests	86.24% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.