You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit f400d5b
Browse filesBrowse the repository at this point in the historyBrowse files
fix(transcript): load large tool images on demand instead of inline (#1118)
A session whose history holds image tool results (for example `read` on
PNG files) never opened over a slow link (#1113): the 200-message page
was 16.8 MB, 15.7 MB of it base64 in five tool results. It took 20 s at
6.5 Mbit/s, and a reverse proxy cut the transfer, so the UI showed
"Failed to fetch". Every open, reload and history window paid for every
image, shown or not.
Transcript pages now carry large tool images by reference. The UI's
native message list reads (createInstanceFetch) and history windows send
the `x-codenomad-media: deferred` opt-in. For those reads only, the
server replaces each tool image whose data URI exceeds 32 KiB with a
`/api/workspaces/<id>/session-media/<session>/<message>/<part>/<index>/
<sha256>` path, in the instance proxy (forwardRuntimeRequest gains an
optional whole-JSON transform for successful JSON responses) and in the
history window route. toolImageSource resolves that path on the API
base, so the existing lazy `<img>` loads only the images that are shown,
as binary rather than base64.
The new session-media route checks that the workspace owns the session,
reads the native message, and refuses a session that moved during the
read and messages at or after an undo boundary. It serves the bytes only
when the image still matches the digest, as an immutable, nosniff,
sandboxed image; anything else falls back to "Unable to display this
image".
What stays whole:
- Single-message reads (`session.message.get`), so pruning revisions
keep hashing native content.
- Live events, user attachments, small images, non-image files and
error responses.
- Requests without the opt-in.
The opt-in header is never forwarded to OpenCode. On the reported
session, the page drops from 16.8 MB to about 1.1 MB.
Validation:
- session-media.test.ts: only large renderable tool images change, and
only their URI.
- routes/session-media.test.ts: exact bytes and headers; invalid
references, digest and position mismatches, the undo boundary,
ownership and a move during the read.
- proxy.test.ts: the transform applies to successful JSON only and
releases once.
- instance-proxy.test.ts: an opted-in list read is deferred, the opt-in
is not forwarded, and plain list or single-message reads stay whole.
- session-navigation.test.ts: history windows.
- sdk-manager.test.ts: only list reads opt in, keeping other headers.
- tool-content.test.ts: path resolution, forged paths rejected; it is
now listed in the PR build.
- tool-images.test.ts: a deferred image decodes from the route in the
real ToolCall renderer, and a missing one shows the fallback.
The UI own-tsconfig, server and root typechecks pass, as do:
- the server suite (2,467 passed; one index.test.ts process test timed
out under load and passes alone, as on dev);
- the UI unit step of the PR build (564 tests);
- seven browser suites (278 tests).
- Status Tokens are independent of message token visibility. A collapsed section uses the non-interactive `renderSummary` slot for one ellipsized session input/output/cost line; expanded content shows session metrics then descendant-inclusive totals as separate wrapping groups. Keep this layout in `styles/panels/status-tokens.css`.
10
10
- Message token disclosures keep one row in both states: collapsed shows the input/output/cost summary chips on a single ellipsized line; expanded swaps the same row to all six chips beside the chevron, which wrap onto further lines when narrow instead of truncating (expansion exists to read every value). Keep the metric renderer local and chip/row styles in `styles/messaging/message-usage.css`.
11
11
- Tool-result images retain native `state.content` and render through the shared `components/tool-call/output-images.tsx` surface, including MCP and specialized tools. Keep image bytes out of text projections (copy/search/speech); image styles live in `styles/messaging/tool-call/images.css`.
12
+
- Transcript pages carry large tool images by reference: the UI's native message list reads (`lib/sdk-manager.ts`) and history windows (`fetchHistoryWindow`) send the `x-codenomad-media: deferred` opt-in (`server/src/opencode/session-media-contract.ts`), and the server replaces each tool image data URI over `INLINE_MEDIA_LIMIT` (32 KiB) with a `/api/workspaces/<id>/session-media/<session>/<message>/<part>/<index>/<sha256>` path (`opencode/session-media.ts`). `toolImageSource` resolves that path on the API base, so the lazy `<img>` loads only shown images; `routes/session-media.ts` serves the bytes of the owned session's native message, pinned by the digest and refused at or after an undo boundary, as an immutable sandboxed image. Single-message reads (`session.message.get`: pruning revisions, previews), live events, user attachments and requests without the opt-in stay whole, so revisions hash native content.
12
13
- Code Mode reads native `metadata.toolCalls` in `tool-call/renderers/execute-data.ts`; keep call-list styling in `styles/messaging/tool-call/execute.css`, with native image/output ownership in the shared tool shell.
13
14
- Native websearch text is parsed conservatively by `tool-call/renderers/websearch-data.ts`; unknown/oversized responses use the shared raw renderer. Recognized provider-consent Forms retain the generic schema/answer path, with layout in `styles/messaging/tool-call/websearch.css`.
14
15
- Session lifecycle features use OpenCode V2's published signals: one `session.execution.{succeeded,failed,interrupted}` per busy period (`lib/native-execution-events.ts`) and `session.retry.scheduled` for provider backoff. The schema-declared `session.idle`/`session.status` are never published; never wire a feature only to them. A terminal event is a hint until `session.active` confirms settlement, which alone announces "A session finished", only for root sessions ending in success, failure or an inactivity stop. Transcript reducers retire at settlement, deferred while prompts stay parked in their inbox or a user shell or unsettled tool call still needs reconciling there. Token/cost lines (`components/message-usage.tsx`) appear once per agent message: the row of an assistant message's final text (the one carrying "delete tools and reasoning") renders, once that message has native usage, the sum (`sumMessageUsage`) of its own usage and of every message holding a tool or reasoning part that cleanup selects (`technicalCleanupParts`). Reuse that selection rather than a separate projection. The line renders inside the content row, so it never shifts `<Index>` items or splits a technical group (whose Delete needs two tools); tool-only steps get no line. A message whose text is followed by tools also counts in the next message's line, as its tools belong to that selection. Sums cover the loaded transcript window only, like cleanup. Capture real streams with `scripts/test-session-lifecycle-native.mjs`.
0 commit comments