Skip to content

Add provider-level tool_use_kcount for prices shared across all models #291

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Kludex wants to merge 1 commit into
base: add-web-search-request-pricing
Choose a base branch
from
Open

Add provider-level tool_use_kcount for prices shared across all models #291

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion packages/js/src/api.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ export function calcPrice(usage: Usage, modelId: string, options?: PriceOptions)
if (!model) return null
const timestamp = options?.timestamp ?? new Date()
const modelPrice = getActiveModelPrice(model, timestamp)
const priceResult = calcPriceInternal(usage, modelPrice)
const priceResult = calcPriceInternal(usage, modelPrice, provider.tool_use_kcount)
Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration bot Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Provider-level tool_use_kcount lost when model is resolved via fallback provider

When a model is found through the fallback_model_providers mechanism (e.g., Azure falling back to OpenAI for gpt-4o), the code passes the requesting provider's tool_use_kcount — not the fallback provider's. Since Azure and Google don't define tool_use_kcount, all provider-level tool pricing moved by this PR is silently dropped for fallback lookups.

Detailed explanation of the fallback pricing loss

Azure has fallback_model_providers: ['openai', 'anthropic'] (packages/js/src/data.ts:1517) and does not define its own gpt-4o model. When a user queries Azure for gpt-4o, the matchModelWithFallback function finds OpenAI's gpt-4o model definition. However, the provider variable at packages/js/src/api.ts:73-74 remains the Azure provider (which has no tool_use_kcount).

Before this PR, OpenAI's gpt-4o model had tool_use_kcount: {web_search: 25, file_search: 2.5} at the model level, so fallback worked correctly. After this PR, file_search: 2.5 was moved to OpenAI's provider-level tool_use_kcount, leaving the model with only {web_search: 25}. Since provider.tool_use_kcount is Azure's (undefined), file_search pricing is completely lost.

The same issue applies to:

  • Azure → Anthropic fallback: Claude models lose web_search: 10 pricing
  • Google → Anthropic fallback: Claude models lose web_search: 10 pricing

Impact: Users calculating prices through Azure or Google providers for fallback models will get incorrect (lower) total_price values — file_search and web_search tool use costs will be $0 instead of the correct amount.

Prompt for agents 
The fix needs to handle the case where a model is resolved via fallback_model_providers. When the model comes from a fallback provider, the provider-level tool_use_kcount should come from the fallback provider, not the original requesting provider.

In packages/js/src/api.ts line 80, instead of always using provider.tool_use_kcount, the code needs to determine which provider actually supplied the model. One approach:
1. Modify matchModelWithFallback in packages/js/src/engine.ts (around line 219) to also return which provider the model was found in.
2. Use that provider's tool_use_kcount in the calcPriceInternal call at packages/js/src/api.ts line 80.

The same fix is needed in the Python side:
- In packages/python/genai_prices/data_snapshot.py around line 59-65, the calc method passes the original provider to model.calc_price. When the model was found via fallback, the fallback provider's tool_use_kcount should be used instead.
- One approach: modify find_provider_model to also track which provider the model was actually found in, and pass that provider's tool_use_kcount.

Alternatively, a simpler fix would be to keep file_search and web_search at the model level (reverting the data change) while still supporting provider-level tool_use_kcount for future use cases where fallback is not involved.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

return {
auto_update_timestamp: undefined,
model,
Expand Down
59 changes: 6 additions & 53 deletions packages/js/src/data.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,9 @@ export const data: Provider[] = [
],
},
],
tool_use_kcount: {
web_search: 10,
},
models: [
{
id: 'claude-2',
Expand Down Expand Up @@ -120,9 +123,6 @@ export const data: Provider[] = [
cache_write_mtok: 1,
cache_read_mtok: 0.08,
output_mtok: 4,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand All @@ -146,9 +146,6 @@ export const data: Provider[] = [
cache_write_mtok: 3.75,
cache_read_mtok: 0.3,
output_mtok: 15,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -178,9 +175,6 @@ export const data: Provider[] = [
cache_write_mtok: 3.75,
cache_read_mtok: 0.3,
output_mtok: 15,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand All @@ -196,9 +190,6 @@ export const data: Provider[] = [
cache_write_mtok: 0.3,
cache_read_mtok: 0.03,
output_mtok: 1.25,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand All @@ -215,9 +206,6 @@ export const data: Provider[] = [
cache_write_mtok: 18.75,
cache_read_mtok: 1.5,
output_mtok: 75,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand All @@ -234,9 +222,6 @@ export const data: Provider[] = [
cache_write_mtok: 3.75,
cache_read_mtok: 0.3,
output_mtok: 15,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -265,9 +250,6 @@ export const data: Provider[] = [
cache_write_mtok: 1.25,
cache_read_mtok: 0.1,
output_mtok: 5,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -296,9 +278,6 @@ export const data: Provider[] = [
cache_write_mtok: 18.75,
cache_read_mtok: 1.5,
output_mtok: 75,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand All @@ -321,9 +300,6 @@ export const data: Provider[] = [
cache_write_mtok: 18.75,
cache_read_mtok: 1.5,
output_mtok: 75,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -352,9 +328,6 @@ export const data: Provider[] = [
cache_write_mtok: 6.25,
cache_read_mtok: 0.5,
output_mtok: 25,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -415,9 +388,6 @@ export const data: Provider[] = [
},
],
},
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -449,9 +419,6 @@ export const data: Provider[] = [
cache_write_mtok: 3.75,
cache_read_mtok: 0.3,
output_mtok: 15,
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -506,9 +473,6 @@ export const data: Provider[] = [
},
],
},
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -563,9 +527,6 @@ export const data: Provider[] = [
},
],
},
tool_use_kcount: {
web_search: 10,
},
},
},
{
Expand Down Expand Up @@ -8609,6 +8570,9 @@ export const data: Provider[] = [
],
},
],
tool_use_kcount: {
file_search: 2.5,
},
models: [
{
id: 'ada',
Expand Down Expand Up @@ -8984,7 +8948,6 @@ export const data: Provider[] = [
output_mtok: 8,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand All @@ -9010,7 +8973,6 @@ export const data: Provider[] = [
output_mtok: 1.6,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand All @@ -9036,7 +8998,6 @@ export const data: Provider[] = [
output_mtok: 0.4,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9082,7 +9043,6 @@ export const data: Provider[] = [
output_mtok: 10,
tool_use_kcount: {
web_search: 25,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9127,7 +9087,6 @@ export const data: Provider[] = [
output_mtok: 0.6,
tool_use_kcount: {
web_search: 25,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9284,7 +9243,6 @@ export const data: Provider[] = [
output_mtok: 10,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9333,7 +9291,6 @@ export const data: Provider[] = [
output_mtok: 2,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand All @@ -9358,7 +9315,6 @@ export const data: Provider[] = [
output_mtok: 0.4,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9431,7 +9387,6 @@ export const data: Provider[] = [
output_mtok: 10,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9461,7 +9416,6 @@ export const data: Provider[] = [
output_mtok: 2,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down Expand Up @@ -9510,7 +9464,6 @@ export const data: Provider[] = [
output_mtok: 14,
tool_use_kcount: {
web_search: 30,
file_search: 2.5,
},
},
},
Expand Down
17 changes: 14 additions & 3 deletions packages/js/src/engine.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,11 @@ function calcMtokPrice(
return calcTieredPrice(price, tokens, totalInputTokens)
}

export function calcPrice(usage: Usage, modelPrice: ModelPrice): ModelPriceCalculationResult {
export function calcPrice(
usage: Usage,
modelPrice: ModelPrice,
providerToolUseKcount?: Record<string, number>
): ModelPriceCalculationResult {
let inputPrice = 0
let outputPrice = 0

Expand Down Expand Up @@ -94,8 +98,15 @@ export function calcPrice(usage: Usage, modelPrice: ModelPrice): ModelPriceCalcu
if (modelPrice.requests_kcount !== undefined) {
totalPrice += modelPrice.requests_kcount / 1000
}
if (modelPrice.tool_use_kcount && usage.tool_use) {
for (const [unit, price] of Object.entries(modelPrice.tool_use_kcount)) {
const effectiveToolUseKcount: Record<string, number> = {}
if (providerToolUseKcount) {
Object.assign(effectiveToolUseKcount, providerToolUseKcount)
}
if (modelPrice.tool_use_kcount) {
Object.assign(effectiveToolUseKcount, modelPrice.tool_use_kcount)
}
if (Object.keys(effectiveToolUseKcount).length && usage.tool_use) {
for (const [unit, price] of Object.entries(effectiveToolUseKcount)) {
const count = usage.tool_use[unit] ?? 0
if (count) {
totalPrice += (price * count) / 1000
Expand Down
1 change: 1 addition & 0 deletions packages/js/src/types.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,6 +105,7 @@ export interface Provider {
price_comments?: string
pricing_urls?: string[]
provider_match?: MatchLogic
tool_use_kcount?: Record<string, number>
}

export interface ModelPriceCalculationResult {
Expand Down
Loading