Repository navigation
[Bug] OpenAI-compatible providers never request streaming usage, so Ollama and similar servers report no tokens #1658
Description
Description
createOpenAICompatible(...) in source/ai-sdk-client/providers/provider-factory.ts (lines 532-538) is created without includeUsage: true, so the SDK never sends stream_options: {include_usage: true}. Ollama and some other servers only send token usage in a streamed reply when asked, so Nanocoder gets none. The per-response usage indicator falls back to estimates, and real token counts, cost and cache-hit data are missing. The --json report from run has no usage, and the agent eval harness records 0 tokens for local models (benchmarks/agent/harness.ts:441-443 uses usage.totalTokens ?? 0).
Environment
- OS: Windows 11 (10.0.26300)
- Node version: 22.22.3
- Nanocoder version: 1.31.0 (main at 5c2e693)
- Provider: Ollama 0.35.1 (OpenAI-compatible endpoint)
- Model: qwen2.5:1.5b
Steps to Reproduce
- Configure an Ollama provider.
- Put a small logging proxy between Nanocoder and Ollama (or inspect the request another way).
- Run
nanocoder --trust-directory --json run "Reply with the single word OK". - Look at the request body and at the JSON report.
- As a control, send the same request to Ollama by hand, once with
"stream_options":{"include_usage":true}and once without.
Expected Behavior
The request contains "stream_options": {"include_usage": true}, and the report includes token usage.
Actual Behavior
The request keys are only model, messages, tools, tool_choice, stream, and the streamed reply has no usage. The report has no usage field:
keys: kind, exitCode, finalText, reasoning, toolCalls, steps, filesChanged
usage: undefined
The control request shows the server does send usage when asked:
with stream_options: "usage":{"prompt_tokens":31,"prompt_tokens_details":{"cached_tokens":5}, ...
without stream_options: 0 usage chunks
Suggested Fix
- Pass
includeUsage: truetocreateOpenAICompatibleinprovider-factory.ts(the SDK option exists in@ai-sdk/openai-compatible). - Consider a per-provider setting to turn it off, for any server that rejects
stream_options. - Add a test in
provider-factory.spec.ts: start a small local HTTP server, stream one reply through the provider, and check that the request body hasstream_options.include_usage === trueand that the usage comes back. - Related: the harness could store missing usage as
nullinstead of0.
Additional Context
- Related: feat(usage): show API-reported context usage with an estimate fallback #543 (API-reported usage with an estimate fallback), [Feature] Token Usage & Cost Indicator per Response #756 (usage indicator), open PR feat: (P1)track prompt prefix stability and report cache hit rate #1649 (cache hit rate, which needs this data).
- I have searched existing issues to ensure this is not a duplicate
- I can reproduce this issue consistently
- This issue occurs with the latest version of nanocoder
Activity
dhruvil-codes commentedon Oct 9, 2026
Hi @akramcodez @addyCooks, I would like to take this on!
Proposed Approach:
source/ai-sdk-client/providers/provider-factory.ts(insidecreateOpenAICompatible(...)), passincludeUsage: trueso the underlying@ai-sdk/openai-compatibleclient sends"stream_options": { "include_usage": true }on streaming requests.source/ai-sdk-client/providers/provider-factory.spec.tsasserting that streamed requests to OpenAI-compatible providers includestream_options.include_usage === trueand return token metrics.Could you please assign this issue to me? I will have a PR ready shortly.
addyCooks commentedon Oct 9, 2026
Thanks @dhruvil-codes,
assigning this to you for now..
I can see PR #1671 is already up
For future issues, please wait until you've been assigned before opening a PR.
Assignment avoids duplicate work, and it lets us flag design concerns before you've written the code.
Comment on the issue as you did here, then start once you're assigned.
dhruvil-codes commentedon Oct 9, 2026
Thanks @addyCooks for assigning this to me! Understood completely regarding the workflow. I'll be sure to wait for the official assignment before opening PRs on future issues.
Appreciate the guidance!
fix(providers): request streaming usage by default for openai-compati…
fix(providers): request streaming usage by default for openai-compati…