Introduce dbval.store: pluggable tuple storage behind a protocol

[maxweber]

Summary

Extracts the storage layer behind dbval.store/ITupleStore, so the engine (transact, indexes, query, pull) is storage-agnostic. This is step 1+2 of the plan to unify dbval and slateval: the contract is exactly what both backends provide — a transactional ordered key-value store.

Stacked on #6 (conn-without-atom); will retarget to main once #6 merges.

The protocol (deliberately minimal)

(-scan    [store begin end reverse?]) ; committed keys in [begin, end), unsigned byte order
(-commit! [store keys])               ; atomically add a batch of keys
(-close!  [store])

The key design decision: stores never see uncommitted state. dbval's transact engine reads mid-transaction (upsert resolution, retract lookups), which SQLite used to provide via read-your-writes on the shared connection — SlateDB's WriteBatch can't. So that concern moves into the engine: a transaction stages its keys in a pending TreeSet overlay carried by the db value, and slice lazily merges the overlay over the store scan. Consequences:

• nothing touches the store until one atomic -commit! — a failing transaction just discards the overlay, and the JDBC rollback machinery is deleted
• the contract matches SQLite, SlateDB, and FoundationDB naturally (Datomic-style tiny storage contract)

Adapters

• dbval.store.sqlite (default, used when empty-db gets :db-file/no :store): the previous storage code moved. Reads now run with autocommit, so scans always see the latest committed state — this fixes the WAL snapshot pinning where a second connection to the same file never saw other connections' commits. Snapshot consistency comes from :max-tx filtering (the store is append-only), not storage-level read transactions. The previously-removed two-connections test is re-added and passes.
• dbval.store.memory: a ConcurrentSkipListSet — engine tests without any storage backend, and the natural third implementation proving the seam.

No bundled storage driver (da2437f)

org.xerial/sqlite-jdbc moved out of :deps into the :dev/:bench aliases — consumers on the SlateDB or memory backend no longer download the SQLite driver's ~13 MB of bundled natives. The adapter itself stays in the artifact (it only references java.sql and loads the driver reflectively); empty-db resolves it lazily via requiring-resolve, and a missing driver produces an instructive error pointing at org.xerial/sqlite-jdbc / :store, verified on a driver-free classpath. Same model as next.jdbc: you bring the driver for the store you pick. dbval's only remaining hard deps: fdb-java (the tuple codec) and colossal-squuid.

Tests

• Full suite (SQLite default) exercises the overlay implicitly — every transact reads mid-transaction: 162 tests, 1048 assertions, 0 failures.
• New dbval.test.store: transact/upsert/retract-history/query/pull/rseek/index-range/snapshot-isolation and failed-transaction atomicity on the memory store.
• test-deref-sees-other-connections: the cross-connection freshness win, previously impossible.
• Bench smoke on the store-based engine: add-all 415ms, q2 69ms, find-datom 16ms, pull-many 217ms — same ballpark as before.

Follow-ups (not in this PR)

• dbval.store.slatedb adapter + porting slateval's as-of/since/history into the shared engine, then archiving slateval.
• Closeable scans (the statement-leak fix) at the protocol level.
• BEGIN IMMEDIATE-style cross-process write locking as an adapter concern.

🤖 Generated with Claude Code

https://claude.ai/code/session_014dX8tTR4yFh5SyBGw3atpo