[rohanshrma222]

Description

First PR of 6 for #1574 (KV-cache-aware prompt lanes).

Before restructuring prompt construction, we need to be able to see where the
cache actually breaks otherwise there's no way to show that the later PRs
help. This PR adds only measurement; no prompt, tool or history behaviour
changes.

1. Prompt prefix stability tracking (source/ai-sdk-client/chat/prefix-tracker.ts)

Fingerprints each outgoing request — system prompt, tool schemas in send
order, and converted history — and compares it with the previous request on
the same client. One debug line per request reports:

• append-only — only new messages were added, so the server can reuse its cache
• system-changed — the system prompt differed
• tools-changed — which tools were added or removed, or whether they were reordered
• history-changed — the index of the first rewritten message

Only hashes are logged, never prompt content. The whole thing is wrapped in
try/catch so diagnostics can never fail a request, and tool schema hashes are
memoized in a WeakMap, so there's no per-request seria
the first call.

2. Cache hit rate in the usage indicator (source/usage/format.ts)

Tokens: 12.4k | 9.8k cached (82%) | ~$0.02. The percen
than guessed when a provider doesn't report input tokens or reports
inconsistent counts.

Worth noting for reviewers: nanocoder already picks up cached-token counts from
OpenAI-compatible servers (llama.cpp, vLLM, OpenRouter)
just Anthropic — so no provider plumbing was needed here, only surfacing it.

The rest of the plan

Each of these will be its own PR against the same issue:

2. Move per-turn content (relevance-matched skills, reca
   out of the system prompt into the current user turn — currently the system
   prompt differs on every user message, which looks lik
3. Deterministic tool ordering; keep MCP tool schemas registered when a server
   goes unhealthy instead of unregistering mid-session
4. Freeze session-stable context (AGENTS.md, system info
   switch rather than removing them
5. Chunked message capping and prefix-preserving compaction
6. Optional backend hints (cache_prompt / id_slot) and docs for
   --cache-reuse, --enable-prefix-caching, fixed `nu

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update

Changeset

• Added a changeset (pnpm changeset) describing th

Testing

Automated Tests

• New features include passing tests in .spec.ts/tsx files
• All existing tests pass (pnpm test:all completes
• Tests cover both success and error scenarios

14 new tests: 11 for the tracker (append-only, system change, tool
add/remove/reorder, schema change, rewritten history, dr
determinism, concurrent ordering), 1 asserting handleChat records each
request, and 3 for the hit rate.

On pnpm test:all: scripts/test.sh doesn't run on my
each check separately — test:format, test:lint, test:types and test:knip
all pass, and the targeted spec files pass. The full AVA
failures on Windows (path validation, bash executor, daemon Unix sockets, voice
plugin, ink imports); I confirmed the one nearest my change,
source/usage/calculator.spec.ts, fails identically on
a CI run to confirm green on Linux.

Manual Testing

• Tested with Ollama
• Tested with OpenRouter
• Tested with OpenAI-compatible API
• Tested MCP integration (if applicable)

Checklist

• If this was for an open issue, I was assigned to it
• Code follows project style guidelines
• Self-review completed
• Documentation updated (if needed)
• No breaking changes (or clearly documented)
• Appropriate logging added using structured logging (see CONTRIBUTING.md)

[github-actions]

nc-review: comments — 1 nit

@rohanshrma222 — a few things worth a look, none blocking.

The PR adds prompt-prefix observability: a PrefixTracker per AISDKClient that fingerprints outgoing requests (system + tool schemas + history) and reports the first divergence from the previous request via a debug log, plus a getCacheHitRate helper that decorates the existing per-response indicator with (NN%). Both halves are self-contained, the try/catch around the tracker guarantees diagnostics never fail a request, and the tests (11 for the tracker, 1 for the chat-handler integration, 3 for the hit-rate) genuinely exercise the behaviour and would fail under regression. One nit: the debug log is only visible when users set NANOCODER_LOG_LEVEL=debug, which is a real friction point for an observability PR whose headline benefit is visibility.

⚪ nit · design · source/ai-sdk-client/chat/prefix-tracker.ts:0

The prefix-stability signal is logged at logger.debug(...), which is below the production default file level (info, see source/utils/logging/config.ts:83-86) and silent on console by default. The cache-hit-rate percentage is always visible in the indicator, so the headline UX is fine, but the diagnostic that is supposed to explain a low hit rate requires NANOCODER_LOG_LEVEL=debug to see. Either elevate the prefix break log to info (small per-request payload, only emits on divergence — a frequency gate that already filters for interesting cases), document the env-var requirement next to the feature, or add a short summary counter the user can query without the log file.

---

_{🔴 blocking · 🟠 a reviewer would ask for a change · ⚪ optional}

_{Automated code review — correctness, security, design, tests, plus duplicates and scope. A human still decides; this is not a substitute for review and is not exhaustive. The required status checks separately cover lint, formatting, types, unused dependencies, the test suite and the build. This bot never merges. Maintainers can rerun with /re-review.}

[github-actions]

Hi @rohanshrma222, thanks for this PR! It looks like a maintainer has left feedback
or review activity and there are still some outstanding items to wrap up.

Whenever you get a chance, could you take a look at the open comments?
If anything is unclear or you'd like a hand, just reply here and we'll help you get it across the line.

[github-actions]

Hi @rohanshrma222, just a heads-up that this PR still has outstanding items
and hasn't seen any activity in a while.

If we don't hear back within about a week, we'll close it to keep the queue tidy.
You can always reopen it later. Happy to help if anything is blocking you - just say the word.

[rohanshrma222]

@akramcodez

PTAL