Skip to content

feat(chat): token stats — tok/s, TTFT footer stat with settings toggle - #45

Open
1337hero wants to merge 2 commits into
mainfrom
feat/token-stats
Open

feat(chat): token stats — tok/s, TTFT footer stat with settings toggle#45
1337hero wants to merge 2 commits into
mainfrom
feat/token-stats

Conversation

@1337hero

Copy link
Copy Markdown
Owner

Closes #35

What

Optional per-message token stats footer on assistant messages: tokens/sec, TTFT, and token counts. Off by default via a new Settings toggle — when disabled, nothing renders (zero layout shift).

How

  • Server (server/src/routes/chats.js): attaches {usage} as message metadata on the finish part via toUIMessageStreamResponse({ messageMetadata }).
  • Client timing (useChatStream.js): TTFT measured send → first assistant text delta; duration measured client-side. Client composes metadata.stats from usage + timing so timing is measured where it happens.
  • tok/s = outputTokens / (duration − TTFT) — generation rate, excluding TTFT.
  • Persistence: stats ride the existing message-metadata path; messageShape.js now carries metadata through toCanonicalMessage, so stats survive reload.
  • Toggle: showTokenStats in useThemeStore (localStorage, per-browser), matching the showCodeLineNumbers pattern.

Verification

  • bun run test frontend: 52 pass, incl. 7 new tokenStats.test.js tests (null guards, tok/s math, zero-generation guard, formatting)
  • bun test server: 420 pass
  • bun run build, bun run format clean

Decisions to ratify

  • Toggle is client-side (localStorage) rather than server app settings — matches existing display-preference pattern.
  • Not yet checked against a live Ollama vs cloud provider (issue's acceptance check) — worth a manual pass before merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PRD (nice-to-have): Token stats display — tokens/sec, TTFT

1 participant