Skip to content
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
88c0997
feat(store): the entity schema, with the rules in it
lollipopkit Aug 19, 2026
4f8b662
feat(store): carry the metadata an incremental sync needs
lollipopkit Aug 19, 2026
ac90458
feat(store): m004, entities out of kv and into tables
lollipopkit Aug 19, 2026
960554a
test(store): cover every released build as a migration source
lollipopkit Aug 19, 2026
dcde9cb
feat(store): the base an entity store sits on
lollipopkit Aug 19, 2026
430ce8e
feat(store): ServerStore over the server tables
lollipopkit Aug 19, 2026
89b1ff0
build: add drift, on the connection the app already opens
lollipopkit Aug 19, 2026
d4f532a
feat(store): Drift owns the schema
lollipopkit Aug 19, 2026
865cebc
feat(store): the entity stores over their own tables
lollipopkit Aug 19, 2026
776160f
test(store): the whole upgrade path, and the adapters it broke
lollipopkit Aug 19, 2026
a2f0ddc
feat(store): the call sites, and a unique name the user is told about
lollipopkit Aug 19, 2026
ecc085c
docs: the storage layout as it is, not as Hive was
lollipopkit Aug 19, 2026
757820a
build(fl_lib): follow the KvStore rename onto current main
lollipopkit Aug 19, 2026
76558c5
Merge origin/main into refactor/entity-tables
lollipopkit Aug 19, 2026
601fd78
fix(store): review follow-ups on #1322
lollipopkit Aug 19, 2026
f6322ad
fix(store): second review round, and the new string in every language
lollipopkit Aug 19, 2026
66f6fb5
fix(store): a migrated conversation kept the old server id in its pay…
lollipopkit Aug 19, 2026
af9d543
test(store): pin the hand-written m004 seed to what a release wrote
lollipopkit Aug 19, 2026
8944240
docs(test): say only what the fixture assertion covers
lollipopkit Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 17 additions & 10 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@ A `Makefile` wraps most common tasks — run `make help` for the full list. Pref
- `flutter test test/disk_test.dart` - Run specific test file (or `make test-one TEST=test/disk_test.dart`)
- `cargo test --workspace` - Run all Rust tests (parser, FFI shell, monitor)
- SSH e2e (opt-in): set `SBM_E2E_SSH_HOST=<ssh destination or ~/.ssh/config alias>` in the workspace-root `.env`, then `cargo test -p sbm_parser --test ssh_e2e` — uploads the generated script to the remote, runs it, and compares the parsed result against direct command output; silently skipped when unset
- A widget test whose tree writes to a store opens the database **in memory**: `SqliteDb.openInMemory()` in `setUp`, `SqliteDb.close` in `tearDown`, and the store's `forTest()` constructor (which only differs by using a distinct store name). Widgets that persist on their own — the floating Agent writes its mode on every change, panes write their width on every drag — write without the test asking them to, so this applies to more trees than it looks like.
- A widget test whose tree writes to a store opens the database **in memory**: `await openTestDb()` from `test/helpers/test_db.dart` in `setUp`, `SqliteDb.close` in `tearDown`, and the store's `forTest()` constructor. That helper is `openInMemory` + the `foreign_keys` pragma + `createTables`; a store backed by a table finds nothing to read without it. `forTest()` no longer varies the name — the schema fixes it — and exists only so the singleton's list cache is not shared between tests. Widgets that persist on their own — the floating Agent writes its mode on every change, panes write their width on every drag — write without the test asking them to, so this applies to more trees than it looks like.
- This is also why: a `testWidgets` body runs in a fake-async zone, and a real *file* write started there completes on a callback that zone is no longer pumping. Under Hive that left the box's write lock held, `close()` in `tearDown` blocked forever, and `flutter test` waited on the process — one such test hung the whole run with no failure and no output naming the file. An in-memory database has no such lock, but keeping the writes off disk is still the rule.
- **A storage migration keeps a permanent regression test, fed by bytes the release being migrated *from* actually wrote.** Permanent meaning the test is never retired once the migration ships; it runs and finishes on every `flutter test` like any other. It gets one pass over a user's records and is not repeatable — once the marker is written the old data is never read again — so a bug there is silence, not a crash. Seeding through the current adapters proves only that today's code agrees with itself.
- `test/fixtures/hive_v1466/` holds boxes generated by v1.0.1466's own adapters (see its README for the worktree recipe); `test/hive_v1466_migration_test.dart` reads them, `test/hive_import_test.dart` covers the import's own logic (retry, idempotency, per-box progress).
- This is not theoretical. `hive_import_test.dart` seeded `Spi` through the current adapter — typeId 15, SSH nested — while 1466 wrote typeId 3 flat, so the `SpiLegacyAdapter` → `_toSpi` path every upgrading install takes had no coverage. The fixture's first run then exposed `PortForwardConfig` being lost **during import** — the one freezed model with no `.g.dart` and so no generated `toJson`, which `SqliteStore.set` reports by returning `false` rather than throwing. `test/port_forward_store_test.dart` establishes separately that an ordinary save loses it the same way, with no migration involved.
- `test/fixtures/hive_v{1466,1480,1491}/` hold boxes generated by those releases' own adapters (see the shared README for the worktree recipe). `test/hive_release_migration_test.dart` runs **both** steps against each — `HiveImport` and then `KvToTablesMigration` — because the shape between them is one no build ships, and asserting there would prove nothing. `test/hive_import_test.dart` keeps its own scope: the import's mechanics (retry, idempotency, per-box progress), asserted against `kv`, which is all the import produces.
- This is not theoretical, twice over. The fixtures' first run exposed `PortForwardConfig` being lost during import — the one freezed model with no `.g.dart` and so no generated `toJson`, which `SqliteStore.set` reports by returning `false` rather than throwing. Their first run against m004 then exposed four field-name mismatches, each silently dropping a whole store: `pubKeyId` vs `keyId`, `private_key` vs `key`, snake_case in `AgentConversation.toJson`, and generated Hive adapters that could no longer open a box at all.
- Do the same for the next migration, and never regenerate a fixture to make a failing test pass.
- Size the view, not the surface, when a test depends on a breakpoint: `tester.view.physicalSize` + `devicePixelRatio`. `setSurfaceSize` changes what the tree is laid out in but not what `MediaQuery` reports, so a "phone" test written that way silently exercises the desktop rendering.
- `pumpAndSettle` is not usable on a tree containing a text field or another always-scheduling widget: it waits for a frame in which nothing is scheduled, and then gives up after its 10-minute default. Count the frames out with `pump(duration)` instead. `--timeout 30s` keeps any such mistake from costing ten minutes.
Expand Down Expand Up @@ -79,7 +79,7 @@ This is a Flutter application for managing Linux servers with the following key
- `store/` - Local storage implementations over `SqliteStore` (fl_lib)
- `lib/view/` - UI layer with pages and widgets
- `lib/generated/` - Generated localization files
- `lib/hive/` - Hive adapters, kept only so `HiveImport` can read an upgrading install's old boxes. Nothing writes Hive; the whole directory goes when that import does (TODO in the code)
- `lib/hive/` - Hive adapters, kept only so `HiveImport` can read an upgrading install's old boxes. Nothing writes Hive; the whole directory goes when that import does (TODO in the code). `legacy_adapters.dart` holds the types frozen at what a release wrote — see the storage notes below before adding a model back to `@GenerateAdapters`
- `lib/src/rust/` - Generated FRB bindings (do not edit)
- `packages/` - Vendored Dart forks referenced by path from pubspec (dartssh2, xterm, fl_lib, fl_build, etc.), each a submodule. The exception is `packages/webui`, an in-repo Svelte package (`@serverbox/webui`) of shared UI primitives and design tokens, consumed as a `file:` dependency by both `monitor/frontend` and `website/`
- `third_party/ish-arm64` - The iOS Linux engine, a submodule of the `lollipopkit/ShellBox` fork. Not in `packages/` because it is C built by meson and consumed by the Xcode project rather than by pubspec. Which revision builds is the gitlink, not a hash in a script: move it with `git submodule update --remote third_party/ish-arm64` and `git add`. `scripts/build-ish-ios.sh` builds it out of tree into `build/ish/build-<arch>/`, so a build never leaves the submodule dirty
Expand Down Expand Up @@ -179,12 +179,19 @@ ends, or restore a consumer.
- AGAIN, NEVER run code formatting commands.
- USE dependency injection via GetIt for services like Stores, Services and etc.
- Generate all l10n files using `flutter gen-l10n` command after modifying ARB files.
- Storage is SQLite, not Hive. `Store` in fl_lib is `sealed`, so a new backend has to be added there as another `part of 'iface.dart'`, not in this repo.
- Most stores are rows in one shared `kv(store, key, value, updated_at)` table, with `value` as JSON — so `get<T>` returns what `jsonDecode` produced and the `fromObj` hook rebuilds the model. `connection_stats` and `agent_conversation` own real tables instead, because every read of them is a range over one server.
- Enums are stored **by name**, never by index: an index silently changes meaning when a case is inserted, and these values outlive the build that wrote them.
- A write that should not count as a user edit — a migration flag, a restore, device-local bookkeeping — passes `updateLastUpdateTsOnSet: false`. `Stores.lastModTime` is read off those timestamps and decides which side of a sync wins.
- **A value passed to `SqliteStore.set` needs a `toJson`.** That is the kv-backed stores. `set` encodes with `(value as dynamic).toJson()` and answers `false` on failure instead of throwing, so a model without one is dropped silently on every write. Freezed generates it only when the model has a `.g.dart`; `PortForwardConfig` has a hand-written `fromJson` and no `.g.dart`, so its `toJson` is hand-written too and has to be kept in step with it. Under Hive this could not happen — the generated typeId adapter did the encoding.
- The two table-backed stores do not go through `set` and so do not share the requirement in the same form. `connection_stats` writes scalar columns only (`timestamp` as millis, `result` as the enum name) and needs no `toJson` at all; `agent_conversation` writes `json.encode(conversation.toJson())` into its `data` column, so it does need one, but calls it itself rather than relying on `set`.
- Storage is one encrypted SQLite file, not Hive. `KvStore` in fl_lib is `sealed`, so a new key-value backend has to be added there as another `part of 'iface.dart'`, not in this repo.
- **Two shapes, and which one a store uses is a decision.** `setting` and `history` are rows in the shared `kv(store, key, value, updated_at)` table with `value` as JSON — a hundred unrelated preferences with nothing that queries by field, where adding one should stay a one-line change. Everything with relations owns a table: `server`, `private_key`, `snippet`, `port_forward`, `conn_stat`, `agent_conversation` and the child tables hanging off them.
- **Drift owns the DDL and only the DDL** (`lib/data/store/db.dart`). Queries are hand-written and synchronous, because the UI reads a store while building; Drift's API is async and converting ~100 call sites would be the change, not the schema. Drift never opens the connection either — `SqliteDb` does, applies the cipher and the `foreign_keys` pragma (which is *per connection*), and `createTables` hands the live handle over.
- `AppDb.schemaVersion` is 1 and stays there. Version is `SchemaVersion`, because the steps that matter read Hive boxes and remap ids — outside what a Drift migration can say. Two mechanisms advancing one number is the ambiguity this arrangement removes.
- **A primary key is an id, never something the user typed.** A private key's id *was* its name, so renaming one detached every server pointing at it; a snippet was keyed by its name. Both have generated ids and a `UNIQUE` name column now, so a rename is an `UPDATE` — and a `DuplicateNameException` out of `EntityStore.put` is where a collision is found, rather than in whichever dialog last remembered to check.
- **A list or map field is a child table**, not a JSON array in a column: `server_tag`, `server_env`, `server_jump`, `snippet_tag`, `container_host`. A child carries no sync columns and moves with its parent, so editing one stamps the parent — `ContainerStore` writes through `Stores.server.synced.stamp`. A child table's rows come back ordered by its primary key, so the order the user added tags in is not kept.
- `Tables.syncRoots` is what sync moves as a unit. Each root has `updated_at` and `rev` — two edits in one millisecond are indistinguishable by the clock alone — and a delete writes a `tombstone` row, without which a peer reads the absence as an addition and puts the record back. `conn_stat` and `agent_conversation` are deliberately not roots: connecting is not an edit, and a conversation carries terminal output and reasoning.
- Enums are stored **by name**, never by index: an index silently changes meaning when a case is inserted, and these values outlive the build that wrote them. A migration reading an older record has to translate explicitly — `ConnectionStat`'s `@JsonValue`s are snake_case while the column holds the enum's `name`, and three of the five differ.
- A write that should not count as a user edit — a migration flag, a restore, device-local bookkeeping — passes `updateLastUpdateTsOnSet: false` for a kv store, or an explicit `at:` to `SyncedTable.stamp`. `Stores.lastModTime` is read off those timestamps and decides which side of a sync wins.
- **A value passed to `SqliteStore.set` needs a `toJson`.** That is the kv-backed stores. `set` encodes with `(value as dynamic).toJson()` and answers `false` on failure instead of throwing, so a model without one is dropped silently on every write. Freezed generates it only when the model has a `.g.dart`; `PortForwardConfig` has a hand-written `fromJson` and no `.g.dart`, so its `toJson` is hand-written too and has to be kept in step with it.
- The entity stores write columns and do not go through `set`, so they do not share the requirement in that form. They do need `toJson`/`fromJson` for the backup, which `EntityStore` declares.
- **`INSERT OR REPLACE` is wrong on a row with sync columns or children.** It deletes and reinserts, so every column the statement does not name goes back to its default — `rev` to 0, defeating the one thing it exists for — and every `ON DELETE CASCADE` fires, taking the children with it. Use `EntityStore.upsert`, which is `ON CONFLICT DO UPDATE` naming only the data columns.
- **Changing a model that `lib/hive/` still has an adapter for breaks the import.** The generator emits `fields[n] as String` for a non-nullable field, so a box written before that field existed fails to *open* and its whole store is silently left behind. `Snippet` and `PrivateKeyInfo` are frozen types in `lib/hive/legacy_adapters.dart` for this reason and are out of `@GenerateAdapters`; do the same rather than regenerating. `test/hive_release_migration_test.dart` is what catches it.
- USE widgets and utilities from `fl_lib` package for common functionalities.
- Such as `CustomAppBar`, `context.showRoundDialog`, `Input`, `Btnx.cancelOk`, etc.
- You can use context7 MCP to search `lppcg fl_lib KEYWORD` to find relevant widgets and utilities.
Expand Down
65 changes: 59 additions & 6 deletions docs/src/content/docs/development/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,65 @@ Server Box follows clean architecture principles with clear separation between d
- Built-in JSON serialization
- CopyWith extensions for updates

### Local Storage: Hive

- **hive_ce**: Community edition of Hive
- Follow the existing model pattern: most stores use `hive_ce`, while some tracked models still declare `@HiveType` and `@HiveField` explicitly
- Type adapters auto-generated
- Persistent key-value storage
### Local Storage: SQLite

One encrypted file, `store.db`, opened through `package:sqlite3` with the
`sqlite3mc` cipher applied before anything reads it. There are two shapes in
it, and which one a store uses is a decision about whether its records have
relations:

- **`kv(store, key, value, updated_at)`** holds the settings and the history —
a hundred unrelated preferences with nothing that queries by field, where
adding one should stay a one-line change. `value` is JSON, so a value written
here needs a `toJson`; `SqliteStore.set` answers `false` rather than throwing
when it has none.
- **Entity tables** hold the servers, private keys, snippets, port forwards,
connection statistics and agent conversations. Real columns, foreign keys,
`CHECK` constraints and indexes.

Drift owns the DDL (`lib/data/store/db.dart`), and only the DDL: the app's
queries are hand-written and synchronous, because the UI reads a store while
building. Drift never opens the connection — `SqliteDb` does, applies the
cipher and the `foreign_keys` pragma, and hands the live handle over.

Two conventions the key-value layout could not hold:

- **A primary key is an id, never something the user typed.** Renaming a
private key used to detach every server pointing at it, because the key's id
*was* its name. Names are ordinary `UNIQUE` columns now.
- **A list or map field is a child table.** That makes "every server with this
tag" a query rather than a decode of every row, and lets `ON DELETE CASCADE`
clean up after a deleted server instead of six hand-written calls.

`Tables.syncRoots` names the tables that are a unit of sync. Each carries
`updated_at` and `rev`; their children carry neither and move with the parent,
so a tag cannot arrive before the server it belongs to. A delete leaves a row
in `tombstone`, without which a peer reads the record's absence as an addition
and puts it back.

### Storage migrations

`SchemaVersion` tracks the layout; Drift's own `schemaVersion` is pinned at 1
and stays there, because the steps that matter are outside what a Drift
migration can express. Two of them exist:

- `HiveImport` (m003) copies an upgrading install's Hive boxes into `kv`, once
per device. It reads through frozen adapters in
`lib/hive/legacy_adapters.dart` rather than through the live models — adding
a field to a model makes a *generated* adapter unable to read any box written
before it.
- `KvToTablesMigration` (m004) takes those rows apart into the entity tables,
generating ids for the records that were keyed by name and rewriting every
reference to them.

**A storage migration keeps a permanent regression test, fed by bytes the
release being migrated *from* actually wrote.** It gets one pass over a user's
records and is not repeatable, so a bug there is silence rather than a crash.
`test/fixtures/hive_v{1466,1480,1491}/` hold boxes produced by those releases'
own adapters, and `test/hive_release_migration_test.dart` runs both steps
against each. Seeding through the current adapters would only show that today's
code agrees with itself — on its first run that test found four field-name
mismatches, each of which silently dropped an entire store.

## Dependency Injection

Expand Down
Loading
Loading