[futurepaul]

Hermes' own state.db as a source for fragment session import, split out of #744 at the review's request. Stacked on #757 (the export read as typed records, streaming), on #744.

It serves people who ran Hermes elsewhere (a state.db, or a Hermes home). The old Finite's customers don't need it: their export never includes state.db.

What it adds

• fragment session import <state.db | Hermes home> (cli/src/session/hermes.rs). It opens the database read-only, in one read transaction, and follows Hermes v0.21.7:
  • A conversation is a session and the sessions Hermes continued it in after each compression (_CHAIN_STEP_SQL's choice among several). Fork markers are bound to the parent's id, as Hermes' pruning and display lineage bind them, so a continuation that inherits a marker still joins its chain. A reset that compresses later is one conversation.
  • Its kind:
    • cron: a scheduled job's run.
    • subagent: a delegate's or a tool's run, whether its parent is gone or not, per the review. Hermes' picker hides _delegate_from rows, and a deleted or pruned parent leaves them orphaned.
    • chat: a root, a branch, or a reset.
    • Also subagent: a passed-over continuation. Hermes shows it nowhere; it's kept, folded away, because it may hold messages.
  • Its messages are what Hermes shows a person, in row-id order, each once across a compression's copied tail (_dedupe_display_generations). It leaves out rows taken back, model_only rows (including metadata an older Hermes double-encoded) and hidden scaffolding. It keeps summaries, marked as such, and the person's words a summary carried after it.
  • It keeps tool-call ids, and results name their call (call). The Hermes profile becomes agent.
  • It reads at reader 20, above the export's 10. As late a copy from Hermes replaces the export's, which kept a compression's tail twice; the export never replaces Hermes'.
• Bounded by what is kept (changed after the re-review):
  • SQLite hands over only what a conversation keeps of each row:
    • structured content is parsed in place (kept_content, with JSON functions): its text parts and a mark for each medium, never a picture's data;
    • tool calls give their ids, name and the start of their arguments, with their whole length (kept_calls);
    • a tool's output and reasoning come at their cut's worth, with their length.
  • So a screenshot's result reads as Screenshot taken\n[image], and a chat of photos or of long file writes is imported rather than refused.
  • A conversation whose kept rows pass twice the conversation cap is refused alone, as soon as they do.
  • A database of more than 200,000 sessions is refused whole.
  • Each conversation goes to sessions: a person's session archive, and fragment session import (the old Finite's export) #744's spool as it is read. A multi-GB state.db is read in bounded memory.
• Not a platform computer's own. Only when the CLI acts as an agent on a platform computer: any database under its Hermes root is refused ($HERMES_HOME, or the folder holding its profiles/). A person's own Hermes on their laptop, which sets HERMES_HOME too, is theirs to import.
• Checked against what Hermes writes. hermes-v0.21.7.sql holds Hermes' own sessions/messages DDL and the rows its SessionDB wrote. It covers a compression with a copied tail, an image, a summary carrying the next words, a model-only row, a branch, a reset that compressed, a cron run, a delegate, and a pruned delegate. A host test reads it; the e2e imports it. regenerate_the_hermes_fixture (ignored, run by name with HERMES_IMAGE) remakes it in our image when Hermes moves.

Where it lives

In the CLI (Paul, 2026-10-30): it serves anyone with the fragment CLI, including someone moving from a self-hosted Hermes who never runs our image. reader and the committed fixture make a fix reach archives made before it, and make drift from Hermes visible.

Complexity ledger

Adds	Where	Why	Removing it	Blast radius
the Hermes reader	cli/src/session/hermes.rs (~560 lines + tests), hermes-v0.21.7.sql (11 KB fixture)	Hermes users who aren't on the old Finite	delete the file and its two-line hook in mod.rs (source)	the CLI only; the export path is untouched
bundled SQLite	cli/Cargo.toml, crates/e2e/Cargo.toml (already in the lock via core's host side)	read state.db	goes with the reader	CLI about 1 MB larger; builds in the image's Alpine stage (checked earlier); macOS and Linux musl in release.yml (run it before the next CLI tag)
an e2e step	crates/e2e/src/lanes/sessions.rs (the fixture as a state.db)	the real CLI on what Hermes writes	delete the step	none

Tests

• hermes.rs host tests:
  • a compressed chain: copied tail once, summary split, a stale sibling, model-only rows (one double-encoded);
  • kinds: branch, reset, a reset that compresses (a continuation inheriting a marker), idle reset, cron, delegate, tool run, both orphaned;
  • stored content: \0json: with an image, legacy part lists, call ids, a long tool output and reasoning cut, saying how many more there were;
  • what is kept, not what is stored: a screenshot result, 6 photo turns of 3 MiB each, 20 write_file calls of 1 MiB each (cut, saying how much more), a list that is no parts, a string Hermes encoded;
  • too much: a 12 MiB conversation refused alone, and a 200,001-session database refused whole;
  • an older schema, a non-Hermes file, read-only;
  • a platform computer's own databases refused (root, sibling profile, own profile), and none for a person's own Hermes (a pure check, no environment);
  • what Hermes v0.21.7 wrote (the fixture) reads as Hermes shows it.
• e2e sessions (11 passed locally): adds "Hermes' own database is imported as Hermes joins its conversations, by its better read".
• cargo xtask check passes.

🤖 Generated with Claude Code

[alexlwn123]

Fail for this stack layer at 9d24d115800767750f349bebd53b55560304bbea, against #744 at 222c0c1ae07b50cf49cfde5e3477136065f17889.

Blocking:

1. Enforce the conversation read budget while iterating messages (cli/src/session/hermes.rs:617–625). rows.collect::<rusqlite::Result<Vec<_>>>() materializes every message in one Hermes session before held is measured. The per-cell SQL cuts do not bound the number of rows. Importing a large session (including retained compacted/inactive rows) therefore consumes memory proportional to that session and can exhaust memory instead of refusing just that conversation and continuing. --kind does not avoid this read either.

I reproduced this with a temporary host test using one session with 64 one-MiB message rows: the refusal reported 64.0 MiB retained, rather than stopping at the first row past the 16 MiB read budget (17 MiB). The probe was removed afterward. Consume rows incrementally, account for their retained allocation before accumulating them, and abandon the conversation immediately at the budget. Include row overhead/metadata or a row-count bound so empty-content rows cannot bypass the memory bound. Test a single oversized session, a chain crossing the budget, and a following small conversation that still imports.

The existing session tests pass (16 passed, 1 ignored: fixture regeneration). CI check and images check pass; e2e shards and real-Hermes Docker remain pending. I did not run those integration suites locally.

Non-blocking: the SQLite reader is isolated and reuses the existing spool, but the CLI now owns substantial version-specific Hermes semantics. The image-export alternative already discussed in the PR would keep that maintenance coupled to the pinned runtime; this placement concern is not the blocker above.

[alexlwn123]

Fail for this stack layer at ad1dbaae35bf5d02b896c6b4ca3d6c8c9a361b2a, against #744 at 222c0c1ae07b50cf49cfde5e3477136065f17889.

The earlier whole-session collection is fixed: the read budget is now checked during iteration. The row-overhead part of the earlier feedback remains unaddressed (noted below). The new SQL content projection introduces a reproducible data-loss blocker.

Blocking:

1. Plain JSON arrays can truncate the imported conversation (cli/src/session/hermes.rs:524, 537–542, 704–706). For an ordinary message containing ["red", "blue"], json_each yields SQL strings red and blue; passing those to json_type(p.value, '$.type') or json_extract(p.value, '$.text') raises malformed JSON. The message iterator then ends, but read_in still spools the accumulated prefix as a successfully read conversation. A temporary host test with three messages—before, ["red", "blue"], after—produced that diagnostic and archived only before. This affects ordinary user/assistant text and tool outputs containing string arrays, not just malformed databases.

Guard element types before applying JSON-object functions in both kept_content and kept_length, preserving arrays that are not typed content parts as plain text. Also refuse the affected conversation on a SQLite iteration failure instead of publishing a partial prefix. Cover string and mixed arrays, genuine structured parts, and a read failure followed by another importable conversation.

Non-blocking: held still counts only content, calls and reasoning, excluding MessageRow storage and its other strings; empty-content rows can accumulate without advancing the budget. Add a row-count/allocation bound to complete the earlier bounded-memory feedback. The reader remains isolated and reuses the spool; its version-specific SQL projection is the main new maintenance cost.

Validation: cargo test -p fragment-cli session:: -- --nocapture passes (17 passed, fixture regeneration ignored). The added three-message reproduction fails and was removed afterward; the checkout is clean. CI check and images check pass; e2e shards and real-Hermes Docker are pending. Those integration suites were not run locally. The referenced engineering-style file is absent on this machine.

[alexlwn123]

Fail for this stack layer at 9074270b25be5ec7ce80dbc20cebfeaf27e2eae6, against claude/session-archive-export-stream at 63c309f1b6451804ff2971c2dbbccfb560c287ee.

The prior JSON-array/partial-import blocker remains: hermes.rs is unchanged from my previous reviewed head. The earlier whole-session collection fix remains intact.

Blocking:

1. Ordinary JSON-array text truncates a conversation (cli/src/session/hermes.rs:524, 537–542, 704–706). json_each yields SQL strings for elements of ["red", "blue"]; the object JSON functions applied to those values raise malformed JSON. The iteration error is logged, but the accumulated prefix is still spooled as a successful conversation. On this head, a temporary host test with before, ["red", "blue"], after again archived only before. This affects user/assistant text and tool output containing string arrays.

Guard element types in both kept_content and kept_length, preserving non-part arrays as plain text. Refuse the entire affected conversation on query/iteration failure, including failures in a compression continuation, instead of publishing a partial prefix. Cover string/mixed arrays, genuine structured parts, and an affected conversation followed by another valid one.

Non-blocking: the previously noted allocation accounting gap remains (held omits row overhead and other strings). The isolated reader reuses the spool appropriately; the version-specific SQL projection remains its main complexity cost.

Validation: existing session tests pass (20 passed, fixture regeneration ignored). The temporary three-message regression test fails (1 archived message versus 3 expected); removed afterward, checkout clean. CI check, images check, e2e shards and real-Hermes Docker were pending when inspected. Full integration suites were not run locally; the referenced engineering-style file is absent on this machine.

[alexlwn123]

Fail for this stack layer at 03c9a57491999d4bb94d295155dac8aeea4b2f90, against #757 at 31e8cdf76c2888c4f8627ac42e398bc7505841af.

The previous JSON-array/partial-import blocker remains: cli/src/session/hermes.rs is unchanged from my last reviewed head. The earlier incremental read-budget fix remains intact.

Blocking:

1. Ordinary JSON-array text truncates a conversation (cli/src/session/hermes.rs:524, 537–542, 704–706). For ["red", "blue"], json_each yields SQL strings; the JSON-object functions applied to those values raise malformed JSON. The iteration error is recorded, but read_in still spools the accumulated prefix as a successful conversation. A temporary host test on this head with before, ["red", "blue"], after archived only before (1 message instead of 3). Ordinary user/assistant text and tool output can trigger this.

Guard element types in both kept_content and kept_length, preserving non-part arrays as plain text. Refuse the entire affected conversation on query/iteration failure, including a failure in a compression continuation, rather than publishing a partial prefix. Cover string/mixed arrays, genuine structured parts, and a failed conversation followed by a valid one.

Non-blocking: the previously noted allocation accounting gap remains: held excludes row storage and other strings, so empty-content rows do not advance the budget. The reader appropriately reuses the existing spool; its version-specific SQL projection remains the principal complexity cost.

Validation: ran cargo test -p fragment-cli session:: -- --nocapture with the temporary regression test: all 22 existing tests passed, fixture regeneration was ignored, and the added reproduction failed as above. Removed the probe afterward; checkout clean. CI images check passed; check, e2e shards and real-Hermes Docker were pending when inspected. Full integration suites were not run locally. The referenced engineering-style file is absent on this machine.

[futurepaul]

Fixed at 49f93f9 (cli/src/session/hermes.rs changed). Point by point:

1. Ordinary JSON-array text truncated a conversation.

• Element types are guarded. A new typed_parts(j) decides whether a JSON list is typed parts: not empty, and every element an object whose type is a string. The object check sits inside a CASE (CASE WHEN p.type = 'object' THEN json_type(p.value, '$.type') END), because SQLite evaluates CASE in order and needn't short-circuit AND/OR. So a string element never reaches an object function.
  • kept_content and kept_length both use it. kept_length is now a nested CASE, so json_type never sees invalid JSON.
  • A list that isn't typed parts (strings, mixed, empty, nested) is kept as the text it is.
  • kept_calls keeps only object entries (WHERE c.type = 'object', applied before the row's values are made).
• A failed read refuses the whole conversation. A query or iteration failure in any session of a conversation, the root's or a compression continuation's, now refuses that conversation whole ("it could not be read whole (session …: …)"). It's never spooled as a prefix, and the next conversation is read. With nothing left to report, the reader returns () instead of a problems list.
• Tests, each failing on the previous head:
  • json_lists_that_are_no_parts_are_kept_as_text covers before, ["red", "blue"], a mixed list, [], a tool output [1, [2], null], genuine \0json: typed parts with an image (read as look\n[image]), a call list holding a string next to a real call, and after. All 8 messages are kept.
  • a_conversation_that_cannot_be_read_whole_is_refused_whole: a view over the messages raises for one row in a continuation. The conversation is refused whole and the following conversation imports.

Non-blocking, row accounting: done too. Each row now counts size_of::<MessageRow>() plus every string it read (role, content, call id, calls, tool name, reasoning, display kind) against the bound. So empty rows can't get past it, and the bound acts as a row limit too. Test: empty_rows_count_toward_what_a_conversation_holds, a session with more empty rows than the bound has room for, is refused.

Placement: Paul decided the reader stays in the CLI (option A). The PR body now says so.

cargo test -p fragment-cli session passes (26, plus the ignored fixture regeneration), and clippy is clean.

[alexlwn123]

Pass for this stack layer at 49f93f9afde3346e47a13a44c8d9ecf5a53aec03, against #757 at 2ba6eb983d34a3febb8cd2d3315fc984175a38cd.

The earlier JSON-array/partial-import blocker is fixed: both content projections guard element types, non-part arrays remain text, and a query or iteration failure refuses the whole conversation, including its compression continuations, before anything is spooled. The earlier incremental-budget fix remains intact; row storage and all retained strings now count too, resolving the empty-row accounting feedback.

The reader adds Hermes databases to the existing import/spool path without changing the archive format. Its version-specific SQL remains the main maintenance cost, but this correction reuses one typed-parts predicate across both projections and stays within the reader. I found no blocking regression in this layer.

Validation: cargo test -p fragment-cli session:: -- --nocapture passed (25 passed, fixture regeneration ignored), including string/mixed/empty arrays, genuine content parts, a failing continuation followed by a valid conversation, and empty-row budgeting. CI check and images check pass; e2e shards and real-Hermes Docker were pending when inspected. Full integration suites were not run locally. The referenced engineering-style file is absent on this machine.