[futurepaul]

Media, importer side: the files a message held come into the Session Archive with it, when the store carries their bytes. Stacked on #752.

What it adds

• Format, add-only. A message's attachments: [{name, media_type, size, sha256, path}]. The bytes are the archive's path, media/<sha256>.<ext>:
  • kept once per file, however many messages hold it;
  • one of 1 MiB or more is a blob, as any fragment's file is;
  • a file is at most 25 MiB (MEDIA_FILE_MAX_BYTES); a larger one is left out and its message kept;
  • each conversation's entry counts its files' bytes (media) toward the archive's 1 GiB (ARCHIVE_MAX_BYTES).
  • The extension comes from the name, so the archive's page serves each file as its kind.
  • A message with a file and no words counts as said.
• The export reader takes attachments from the export's messages, with path under chats/:
  • each file is read as every export file is (regular, inside chats/, bounded) and checked against its sha256 and size;
  • one that is wrong, missing or outside is left out of its message, and said;
  • this is the exporter change proposed to Alex (~/dev/finite/audit/session-import-for-alex.md).
• The writer uploads the files the archive lacks first, then the conversations that name them, so a committed conversation never names a missing file. A replay sends no file again.
• The reader page shows a picture inline with its message and links any other file (download).
• Hermes' state.db (sessions: Hermes' own state.db as a source for fragment session import #750) carries no files, and won't. v0.21.7 writes rows through a text-only projection: an image becomes [screenshot], with [Image attached at: <path>] in the text. Its gateway caches are cleaned after 24 hours, so the bytes are almost never there. Those messages keep their marks.

Complexity ledger

Adds	Where	Why	Removing it	Blast radius
Attachment, media_path, Entry.media	crates/core/src/session.rs (~45 lines + test)	a file kept once, by its hash, counted	drop the field (add-only: older readers ignore it)	the format only
the spool's files, the export's attachments	cli/src/session/mod.rs (add_media, upload before conversations), chats.rs (media)	the files come with their conversations, bounded and checked	delete them	the import only
the reader's files	templates/sessions/site/sessions.js (~30 lines), sessions.css (3 lines)	the person sees the picture they sent	delete them	the archive's page

Tests

• core:
  • a file is kept at its path by its hash, and one with no hash, an uppercase hash, too large, no name or a control character is dropped;
  • a picture-only message counts as said, and its bytes count toward the entry.
• cli:
  • the export's files come with their messages, each kept once; a wrong hash, a missing file and a path outside chats/ are each left out and said;
  • the writer sends the file and the conversations in one commit and the indexes in another, and a later import sends no file again;
  • bytes that aren't what the attachment names are refused.
• e2e:
  • sessions: a picture a message held is kept in the archive by its hash;
  • sessions-ui: the picture loads with the person's message in the reader.
• cargo xtask check passes.

🤖 Generated with Claude Code

[alexlwn123]

Fail for now at 36ff9429d6569c579bab536d9c342fd326c97826, reviewing this layer against #752 at a01cddcc6c6bd7219bc4065884f201b585c2f79e.

The media format and reader fit the existing fragment/file primitives, and the implementation is appropriately scoped. Two importer boundaries need fixing.

Blocking:

1. Media bypasses conversation selection. cli/src/session/chats.rs:127 calls media() before Spool::add() applies --kind, validates the conversation, and selects the newer copy. cli/src/session/mod.rs:373 then uploads every spooled media path, independently of accepted conversations. An import with --kind cron uploads attachments from excluded chats; conversely --kind chat reads/counts/uploads excluded cron files. Enough excluded media can also abort an otherwise valid filtered import at the byte limit. Refused conversations and superseded copies have the same orphan-media path. A temporary regression test using the existing export fixture and a picture on its excluded chat fails: the chat entry is absent but its media remains in the upload spool. Apply selection before media side effects and scope uploads/accounting to retained attachments; cover filtering, refusal and superseded copies.

2. A media-enriched read does not update an already imported conversation. The export reader still uses READER = 10 (cli/src/session/chats.rs:33), the same value as the pre-media reader. With unchanged message timestamps, Entry::newer and session::merge discard the conversation even though its new JSON contains attachments. The media bytes upload, but the archived message never references or displays them. I reproduced this by importing a fixture without attachments, adding a valid attachment without changing timestamps, and merging the next import: no conversation write is selected. This also affects retrying an import after restoring a previously missing attachment file. Advance the reader revision for this improved read and handle attachment recovery on replay without allowing older history to replace newer history. Cover the previous-reader upgrade, missing-file recovery, and an unchanged replay.

Validation: clean-head cargo test -p fragment-cli -p fragment-core session:: passes (27 passed, one fixture-regeneration test ignored). Both temporary regression tests above fail as described and were removed afterward. CI check and images check are green; e2e shards and real-Hermes Docker checks were still pending. I did not run e2e locally. The external engineering-style file named by AGENTS.md was unavailable on this machine.

[alexlwn123]

Fail for now at 40bef7e0b0e3288a5323dc4d8d52fcce1ac4c8ff, reviewing this layer against #752 at 056ba2f0cd0ff7f30de46377ab2d68b7952c1288.

Both blockers from my previous review remain. This head merges the updated base, but does not fix the media selection or replay boundaries. The media format and reader remain appropriately scoped to existing fragment/file primitives.

Blocking:

1. Media still bypasses conversation selection. cli/src/session/chats.rs:176 calls media() before Spool::add() filters --kind, validates the conversation, or chooses the newer copy. add_media() counts and spools those bytes immediately, and write() uploads every missing path in spool.media, regardless of retained conversations. Thus --kind cron still uploads excluded chats' attachments; refused or superseded conversations can also leave orphan media, and excluded media can exhaust the import byte budget. Apply selection before media side effects and restrict accounting/uploads to retained attachments, covering filters, refusal and superseded copies.

2. Attachment enrichment still selects no conversation update. The export's READER remains 10 (cli/src/session/chats.rs:40), while Entry::newer compares only (when, reader) (crates/core/src/session.rs:417). Importing a conversation already archived by the previous reader, or retrying after restoring a missing attachment, leaves its timestamps and reader unchanged: merge() discards the enriched conversation even though its hash differs. Its media uploads but is never referenced/displayed. Advance the reader revision for the improved read and support attachment recovery on replay without replacing newer history with older history. Cover the previous-reader upgrade, missing-file recovery and unchanged replay.

Validation: both temporary regression tests fail on this head: an excluded chat still leaves media in the spool, and adding a valid attachment without changing timestamps selects no conversation write. Removed the temporary tests, then cargo test -p fragment-cli -p fragment-core session:: passed (31 passed, one fixture-regeneration test ignored). CI check and images check pass; e2e and real-Hermes Docker checks remain pending. I did not run e2e locally. The external engineering-style file named by AGENTS.md is unavailable on this machine.

[alexlwn123]

Fail for now at 85277e75afba80329ef5c60004f6a5d93b01d27a, reviewing this layer against #752 at 6b42a1753264287daefb72cffc556900a0717c1f.

Both blockers from my previous review remain. This head incorporates the base's index-recovery fix; it does not change the media selection or attachment-replay boundaries. The media format and reader remain appropriately scoped to the existing fragment/file primitives.

Blocking:

1. Media still bypasses conversation selection. cli/src/session/chats.rs:176 calls media() before Spool::add() filters --kind, validates the conversation, or selects the newer copy. add_media() immediately counts and spools the bytes; write() at cli/src/session/mod.rs:435 uploads every missing spooled media path independently of retained conversations or merge.writes. Consequently, --kind cron uploads excluded chats' attachments; refused and superseded conversations can leave orphan files too, and excluded media can exhaust the import byte budget. Apply selection before media side effects and scope accounting/uploads to retained attachments. Cover filtering, refusal, and superseded copies, including copies rejected against an existing archive.

2. Attachment enrichment still does not update an existing conversation. The export's READER is still 10 (cli/src/session/chats.rs:40), the same as the pre-media reader. Entry::newer compares only (when, reader) (crates/core/src/session.rs:417), and merge() rejects an incoming copy when that pair is unchanged even if its hash differs. Reimporting an already archived conversation with newly available attachments—or retrying after restoring a missing attachment file—uploads the media without updating the conversation to reference/display it. Advance the reader revision for the improved read and support attachment recovery on replay without replacing newer history with older history. Cover the previous-reader upgrade, missing-file recovery, and unchanged replay.

Validation: traced both failures on this head; the export reader and core merge code are unchanged from the previously reviewed head. cargo test -p fragment-cli -p fragment-core session:: passes (33 passed, one fixture-regeneration test ignored). CI check, images check, e2e shards, and real-Hermes Docker checks are pending. I did not run e2e locally. The external engineering-style file named by AGENTS.md is unavailable on this machine.

[futurepaul]

Fixed at 1b4defa: cli/src/session/{chats,mod}.rs and crates/core/src/session.rs changed. Point by point:

1. Media bypassed conversation selection.

• Filter first. A conversation of a kind the import didn't ask for has none of its files read (Spool::wants, checked in chats::read before media()).
• Counting. A file counts toward the archive's bytes only with the conversation the spool keeps. Spool::add counts entry.bytes + entry.media, and add_media no longer counts anything toward the archive.
• Uploads. Spool::attached records the files each kept conversation names. write sends only the files named by the conversations in merge.writes that the archive lacks. So a refused conversation's files never go up, nor those of a copy set aside for a newer one, whether that newer copy is in the same import or already in the archive.
• Disk guard. Files read along the way for refused or superseded copies are bounded separately (SPOOL_MEDIA_MAX_BYTES, twice the archive's bytes), so they can't use up the archive's budget.
• Tests:
  • only_the_files_of_conversations_written_go_up: a kept chat's picture goes up. These do not: an older in-import copy's, a refused conversation's, or the picture on an older copy of a conversation the archive holds newer. spool.bytes counts only what's kept.
  • a_kind_not_asked_for_has_no_files_read: the export read with --kind cron reads no chat picture.

2. A media-enriched read didn't update an existing conversation.

• Reader 11. The export's reader is now 11 (10 read no files), so the media-aware read replaces a reader-10 copy that is as late.
• Recovery. Entry::newer now compares (when, reader, media). As late and from as good a read, the copy holding more of its files wins, which covers an attachment file restored since. An older history still never replaces a newer one, because when is compared first.
• Test a_read_with_more_of_its_files_updates_the_conversation:
  • archived by reader 10, read again by 11 with the picture missing: updated;
  • read again with the picture found: updated, and the file references it;
  • the same read again: unchanged, with no commit;
  • an older copy holding the picture: unchanged.
• Reverting newer to (when, reader) makes it fail at "a file found since updates it".

The README's "An import" section says what goes with what.

One limit, unchanged by this PR: Hermes' own database stores no file bytes, so a later Hermes read (reader 20) replaces an export copy without its attachments. Only the export carries bytes.

cargo test -p fragment-cli -p fragment-core session passes, and cargo xtask check is green.

[alexlwn123]

Pass for this stack layer at 1b4defac549ceb250a18906c5e2cf6a025e8a501, against #752 at 359b96faae2b95f5deee4024fcfa397ec3fa12b9.

Both earlier blockers are fixed: excluded kinds read no media, retained conversations alone count toward the archive, and only conversations selected for writing upload their referenced files. Reader 11 updates pre-media imports; attachment recovery updates an equally recent read, while older histories and unchanged replays remain excluded. The new regression tests cover filtering, refused/superseded copies, previous-reader upgrades, missing-file recovery and replay.

The importer and reader changes remain appropriately scoped to the existing fragment/file primitives. I see no blocking regression in this layer.

Validation: cargo test -p fragment-cli -p fragment-core session:: passed (39 passed, one fixture-regeneration test ignored). CI check and images check pass; e2e shards and real-Hermes Docker checks are still pending. I did not run e2e locally. The external engineering-style file named by AGENTS.md is unavailable on this machine.

[futurepaul]

Merged #752's new head (fdae79e, its origin/master merge) in at ab84ffa, with no force-push and no conflicts. This layer's files (cli/src/session/{chats,mod}.rs, crates/core/src/session.rs, templates/sessions/README.md, templates/sessions/site) are unchanged from the 1b4defa you approved. cargo test -p fragment-cli session passes. Could you re-review this merge head?

[alexlwn123]

Pass for this stack layer at ab84ffae520117c993004bbf17535780926d39c7, against #752 at fdae79e5232069bc8cd303840263ec0eabf60751.

Both earlier blockers remain fixed: media follows retained conversations and selected writes; reader upgrades and recovered attachments update equally recent histories without replacing newer histories or rewriting unchanged replays.

This head merges the updated base. All nine files in this layer are unchanged from my approved head 1b4defac549ceb250a18906c5e2cf6a025e8a501. The implementation remains appropriately scoped to existing fragment/file primitives, and I see no blocking regression in this layer.

Validation: cargo test -p fragment-cli -p fragment-core session:: passed (39 passed, one fixture-regeneration test ignored). CI images check passes; check, e2e shards and real-Hermes Docker checks remain pending. I did not run e2e locally. The external engineering-style file named by AGENTS.md is unavailable on this machine.