haex-crdt

SQLite + SQLCipher storage with column-level LWW CRDT sync (uhlc-based Hybrid Logical Clocks). Extracted from haex-vault; consumed by holzi, which supersedes haex-vault.

Status: v0.3.0. Trait foundation, HLC service, SQL transformers, trigger installer, scanner, cleanup, migration engine, apply pipeline, public Database facade, cross-process file locking, and the plan §8 end-to-end acceptance test all land. Dual-licensed MIT OR Apache-2.0. Consume via haex-crdt = { git = "https://github.com/haexmas/haex-crdt", tag = "v0.3.0" }.

Ownership: source lived in haex-vault. This repository is the extraction target; holzi depends on it. The haex-vault-only surface (the JSON DbConnection layer with execute / execute_with_crdt / select / select_with_crdt, PostWriteHook, the test-shims feature) has been removed; the write path is Database::write, the read path Database::read.

Scope

Provided by this crate:

• Encrypted SQLite storage (SQLCipher).
• A migration engine with two journals — crate-owned CRDT bookkeeping and consumer-owned schema, per plan §4.3.
• HLC service (uhlc-based Hybrid Logical Clocks with SQLite-persisted state).
• Column-level LWW CRDT infrastructure: schema transformer, trigger installer, scanner, apply pipeline.
• Cleanup / retention utilities for deleted-row logs.
• Pluggable DeviceIdProvider — consumers supply the UUID for each logical replica. The provider is authoritative on every open and must return a stable UUID when the same replica reopens the same DB file.
• Pluggable SignatureProvider — consumers add per-column signing without this crate depending on any identity system. NoopSignatureProvider is bundled for consumers that already have an authenticated transport (e.g. an MLS group or an attested iroh channel).
• Pluggable MigrationSource — consumers control where schema migration SQL comes from.

Not provided (intentionally):

• Sync transport (iroh, WebSocket, cloud). Scanner + apply are transport-neutral; wire them up in your consumer.
• Identity, UCAN, or MLS. SignatureProvider is the interface.
• Any Tauri or async framework opinion beyond what rusqlite already implies (blocking I/O).

Writing and reading

db.write(|tx| { ... }) runs one IMMEDIATE transaction on the CRDT write path. Every tx.execute, tx.query_map and tx.query_row goes through the CRDT transformer, so callers never stamp the HLC by hand, and all writes in the closure share one HLC — one transaction group for sync. Tables ending in _no_sync pass through untouched. The closure commits on Ok and rolls back on Err or a panic. The serialized parameters of all writes count against DatabaseConfig::max_transaction_bytes (default MAX_CRDT_TRANSACTION_BYTES); the write that would cross it fails with TransactionTooLarge before it runs. DatabaseConfig::max_value_bytes (default MAX_VALUE_BYTES, SQLite's own default) bounds one string or BLOB value and one row on the connection, also values SQL computes itself (zeroblob, randomblob, printf); a larger one fails with ValueTooLarge before it is allocated. It also bounds the rows of sqlite_master, so it must leave room for the SQL of every table and trigger.

db.read(|connection| { ... }) runs a closure on a read-only query view. The view exposes query_row and query_map; SQLite's query_only mode and an authorizer reject writes, PRAGMA changes, and transaction control, so a read path cannot bypass the CRDT transformer.

db.write_guarded(&guard, |tx| { ... }) and db.read_guarded(&guard, |connection| { ... }) do the same for SQL from a less trusted caller. The SqlGuard carries a SQLite authorizer and an optional progress callback; both are active only while the caller's statement is prepared and stepped, never around the crate's own statements (reading and persisting the HLC, trigger setup, the foreign key check). Trigger bodies are authorized with the trigger name (z_dirty_<table>_*) as accessor. On reads the authorizer is combined with the read-only rule. A denial is SqlGuardDenied, a second statement in the SQL MultipleStatements, and a progress callback returning true interrupts the statement (SqlGuardInterrupted) and rolls the whole transaction back. SqlGuard::max_value_bytes lowers the value limit for the caller's statement (never above DatabaseConfig::max_value_bytes); a larger value is ValueTooLarge. query_with_columns returns the column names also for an empty result.

db.write_guarded_with(&guard, GuardedWriteOptions { schema_mode, local }, ...) is for migrations. In every guarded write a CREATE TABLE or ALTER TABLE of a synced table recreates its triggers in the same transaction (under the new name after RENAME TO). Schema mode switches foreign keys off before the transaction and runs PRAGMA foreign_key_check before the commit; tx.copy_rows_verbatim("INSERT INTO __new_x (...) SELECT ... FROM x") copies a table rebuild with the CRDT columns unchanged and the triggers off, so Drizzle's rebuild sequence neither re-stamps rows nor writes delete markers. Local mode creates tables without CRDT columns and triggers; writes to a table without CRDT columns pass through unstamped, tables with them are still stamped.

rusqlite version contract

write and read take and hand out rusqlite types (ToSql, Row), and db.with_connection(|connection| { ... }) is exposed behind the raw-connection feature (default off). All consumers of haex-crdt in one dependency tree must resolve to the same rusqlite version this crate pins; otherwise Connection's ToSql/FromSql types belong to different crate instances and cannot be passed through the callback. See plan §6.

Usage scope

A Database handle owns one rusqlite::Connection behind an internal Mutex. The intended shape is one Database per DB file per process, shared across threads / async tasks via Database.clone() — the internal Arc makes clones cheap and every clone routes through the same lock.

Opening the same DB file from two processes is rejected. As the very first step of Database::open, an fs2-backed advisory lock is acquired on <path>.lock; a second process (or a second in-process Database::open while a live handle still holds the lock) fails immediately with VaultAlreadyOpenElsewhere. Different DB files remain independently openable, and the OS releases the lock on process exit so a crash cannot strand a database.

Trust contract for NoopSignatureProvider

NoopSignatureProvider performs no transport attestation — it signs with empty payloads and accepts empty incoming signatures. Consumers using it MUST deliver remote changes over an already-authenticated transport. Rejects non-empty incoming signatures it cannot verify, so a downgrade from a real provider on an already-signed vault fails loudly.

Roadmap

Extracted from plan §5. Each slice ends with cargo test green.

• Batch A — trait foundation, HLC service, transformers, error types (this slice).
• Batch B — trigger installer, scanner, cleanup ported and trimmed to the CRDT-generic surface.
• Batch C — database/* core (read shell, write path with PostWriteHook, trigger bootstrap) ported.
• Batch D — migrations engine with two journals (haex_crdt_migrations_no_sync crate-owned, haex_app_migrations_no_sync consumer-owned) and SHA-256 drift detection.
• Batch E — apply pipeline with all-or-nothing signature preflight per plan §4.2.
• Batch F — public Database facade (src/database/) tying DeviceIdProvider, SignatureProvider, MigrationSource and the SQLCipher key together, with install_crdt backfill contract per plan §6.
• Batch F.5 — port haex-vault's vault_lock.rs to src/db/lock.rs; wired into Database::open as the first step so cross-process (and in-process concurrent) opens fail fast with VaultAlreadyOpenElsewhere.
• Batch G — plan §8 end-to-end acceptance test in tests/end_to_end.rs: two Databases on two SQLCipher files, backfill on A, plain install on B, local write → scan → apply → readback. It also verifies provider-authoritative device IDs by reopening each file with the other replica's provider. Gated behind raw-connection since the local write goes through with_connection. The true "consumable from a tagged git commit" check lands with the v0.1.0 tag in Batch H.
• Batch H — dual-licensed MIT OR Apache-2.0, CI on GitHub Actions (fmt + clippy + tests on base and raw-connection feature configs, plus the plan §8 acceptance test), v0.1.0 tag on the merge commit.

Sequence deviation from plan §7

The extraction plan §7 proposes a three-step sequence:

1. Introduce traits inside haex-vault first (in-place refactor, no new repo).
2. Cut a sub-crate under a haex-vault Cargo workspace.
3. Move the sub-crate to a standalone repo, preserving git history via git filter-repo.

This repository skipped Steps 1 and 2 by explicit operator direction. Tradeoff: no git history from haex-vault (clean-start), no continuous-integration safety net during the extraction (haex-vault's tests aren't re-running against the moving code), and trait shapes are validated by the ported tests here rather than by the existing haex-vault test suite. Trait ergonomics may need adjustment when haex-vault and holzi first integrate.

License

Dual-licensed under either of

• MIT license (LICENSE-MIT or https://opensource.org/licenses/MIT)
• Apache License, Version 2.0 (LICENSE-APACHE or https://www.apache.org/licenses/LICENSE-2.0)

at your option. Redistributors must comply with the applicable license terms, including retaining applicable copyright, patent, trademark, and attribution notices and adding prominent notices to modified files as required by Apache-2.0. haex-vault did not have a LICENSE file at extraction time (plan §9 called this a to-be-settled item); this crate settles it here.

Contributions

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual-licensed as above, without any additional terms or conditions.