Skip to content

fix(report): use bare-model fallback in usage-ledger cost estimate#863

Open
JOhnsonKC201 wants to merge 1 commit into
usestrix:mainfrom
JOhnsonKC201:fix/litellm-entry-cost-bare-model-fallback
Open

fix(report): use bare-model fallback in usage-ledger cost estimate#863
JOhnsonKC201 wants to merge 1 commit into
usestrix:mainfrom
JOhnsonKC201:fix/litellm-entry-cost-bare-model-fallback

Conversation

@JOhnsonKC201

Copy link
Copy Markdown

Problem

_estimate_litellm_entry_cost (strix/report/usage.py) builds a fallback list of model-name candidates so the LiteLLM cost lookup can fall back from the prefixed name to the bare model name:

candidates = [model]
if "/" in model:
    candidates.append(model.split("/", 1)[-1])

for candidate in candidates:
    cost = completion_cost(
        completion_response={"model": candidate, "usage": usage_payload},
        model=model,   # pins the lookup to the original name
    )

But the pricing lookup passed model=model, so completion_cost always used the original prefixed name and the bare-name candidate was never actually tried (LiteLLM prioritizes the model= kwarg). The loop's fallback was dead code.

With e.g. STRIX_LLM="openai/mistralai/Mistral-7B-Instruct-v0.3" (pointing LLM_API_BASE at a self-hosted OpenAI-compatible server serving HF-named models), LiteLLM keys the price under the bare Mistral-7B-Instruct-v0.3. Every candidate resolves to a prefixed name absent from the price map, cost stays None, and the entry is reported as $0.00.

The sibling estimator _estimate_response_cost in strix/report/state.py builds the same candidate list and correctly passes model=candidate (locked by test_cost_callback_estimates_cost_with_bare_model_fallback). This path was written with model=model, defeating its own fallback.

Fix

Pass model=candidate so the bare-name fallback is actually attempted, matching _estimate_response_cost. For single-candidate configs (candidate == model) behavior is unchanged.

Tests

Adds two tests to tests/test_cost_tracking.py (mirroring the existing bare-model-fallback test): one on _estimate_litellm_entry_cost, one end-to-end through _estimate_litellm_cost, both pricing only the bare name. uv run pytest tests/test_cost_tracking.py → 11 passed; reverting the fix makes both new tests fail (None == 0.0021).

_estimate_litellm_entry_cost builds a candidate list (prefixed name, then
bare model name) so LiteLLM pricing can fall back to the bare name, but it
pinned the lookup to model=model, so completion_cost always used the
original prefixed name and the fallback candidate was never tried.

Prices keyed under the bare model name — e.g. an openai/<vendor>/<model>
config pointing at a self-hosted OpenAI-compatible endpoint — were dropped
and reported as $0.00. Pass model=candidate, matching the sibling
_estimate_response_cost in report/state.py.
@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR enables bare-model fallback for usage-ledger cost estimates. The main changes are:

  • Passes each model candidate to LiteLLM's explicit model lookup.
  • Adds direct coverage for the entry-level fallback.
  • Adds end-to-end coverage for prefixed OpenAI-compatible models.

Confidence Score: 5/5

This looks safe to merge.

  • No blocking issues found in the changed code.
  • Existing successful lookups remain the first attempt.
  • The new tests cover the corrected fallback path.

Important Files Changed

Filename Overview
strix/report/usage.py Uses the current candidate for each cost lookup while preserving the original model as the first attempt.
tests/test_cost_tracking.py Covers the bare-model fallback through both the helper and aggregate usage paths.

Reviews (1): Last reviewed commit: "fix(report): use bare-model fallback in ..." | Re-trigger Greptile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant