fix(report): use bare-model fallback in usage-ledger cost estimate#863
Open
JOhnsonKC201 wants to merge 1 commit into
Open
fix(report): use bare-model fallback in usage-ledger cost estimate#863JOhnsonKC201 wants to merge 1 commit into
JOhnsonKC201 wants to merge 1 commit into
Conversation
_estimate_litellm_entry_cost builds a candidate list (prefixed name, then bare model name) so LiteLLM pricing can fall back to the bare name, but it pinned the lookup to model=model, so completion_cost always used the original prefixed name and the fallback candidate was never tried. Prices keyed under the bare model name — e.g. an openai/<vendor>/<model> config pointing at a self-hosted OpenAI-compatible endpoint — were dropped and reported as $0.00. Pass model=candidate, matching the sibling _estimate_response_cost in report/state.py.
Contributor
Greptile SummaryThis PR enables bare-model fallback for usage-ledger cost estimates. The main changes are:
Confidence Score: 5/5This looks safe to merge.
Important Files Changed
Reviews (1): Last reviewed commit: "fix(report): use bare-model fallback in ..." | Re-trigger Greptile |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
_estimate_litellm_entry_cost(strix/report/usage.py) builds a fallback list of model-name candidates so the LiteLLM cost lookup can fall back from the prefixed name to the bare model name:But the pricing lookup passed
model=model, socompletion_costalways used the original prefixed name and the bare-namecandidatewas never actually tried (LiteLLM prioritizes themodel=kwarg). The loop's fallback was dead code.With e.g.
STRIX_LLM="openai/mistralai/Mistral-7B-Instruct-v0.3"(pointingLLM_API_BASEat a self-hosted OpenAI-compatible server serving HF-named models), LiteLLM keys the price under the bareMistral-7B-Instruct-v0.3. Every candidate resolves to a prefixed name absent from the price map,coststaysNone, and the entry is reported as $0.00.The sibling estimator
_estimate_response_costinstrix/report/state.pybuilds the same candidate list and correctly passesmodel=candidate(locked bytest_cost_callback_estimates_cost_with_bare_model_fallback). This path was written withmodel=model, defeating its own fallback.Fix
Pass
model=candidateso the bare-name fallback is actually attempted, matching_estimate_response_cost. For single-candidate configs (candidate == model) behavior is unchanged.Tests
Adds two tests to
tests/test_cost_tracking.py(mirroring the existing bare-model-fallback test): one on_estimate_litellm_entry_cost, one end-to-end through_estimate_litellm_cost, both pricing only the bare name.uv run pytest tests/test_cost_tracking.py→ 11 passed; reverting the fix makes both new tests fail (None == 0.0021).