Skip to content

Add cache_write_tokens mapping to UsageExtractor #330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ali-corpo wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add cache_write_tokens mapping to UsageExtractor #330

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions packages/python/genai_prices/data.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -928,6 +928,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
Comment on lines 929 to +933
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Azure’s chat extractor now maps usage.prompt_tokens_details.cache_write_tokens into cache_write_tokens, but Azure/OpenAI-priced models in this file don’t define cache_write_mtok. Since cache_write_tokens are subtracted from uncached input and priced via cache_write_mtok, a non-zero value from the API would undercharge. Consider only extracting this field when a cache_write_mtok is defined (or ensure the pricing model includes the appropriate cache-write rate).

Copilot uses AI. Check for mistakes.
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -1313,6 +1316,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
Comment on lines 1317 to +1321
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deepseek models in this file appear to have cache_read_mtok pricing but no cache_write_mtok. Since cache_write_tokens are priced via cache_write_mtok (and subtracted from uncached input), extracting prompt_tokens_details.cache_write_tokens here can lead to undercharging if the API returns a non-zero value. Consider only mapping this field for providers/models that have a defined cache_write_mtok, or set an explicit cache_write_mtok that matches the provider’s billing semantics.

Copilot uses AI. Check for mistakes.
UsageExtractorMapping(
path=['completion_tokens_details', 'audio_tokens'], dest='output_audio_tokens', required=False
),
Expand Down Expand Up @@ -1393,6 +1399,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['completion_tokens_details', 'audio_tokens'], dest='output_audio_tokens', required=False
),
Expand Down Expand Up @@ -1596,6 +1605,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2247,6 +2259,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2289,6 +2304,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2348,6 +2366,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2426,6 +2447,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2593,6 +2617,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -2914,6 +2941,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -3637,6 +3667,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -3882,6 +3915,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -3984,6 +4020,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -4101,6 +4140,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -4218,6 +4260,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -4705,6 +4750,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(path='completion_tokens', dest='output_tokens', required=True),
],
api_flavor='chat',
Expand Down Expand Up @@ -5005,6 +5053,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
Comment on lines 5054 to +5058
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cache_write_tokens is treated as a separately-priced bucket in ModelPrice.calc_price() (it is subtracted from uncached input and priced via cache_write_mtok). OpenAI model entries in this data file do not define cache_write_mtok, so if responses start including usage.prompt_tokens_details.cache_write_tokens with a non-zero value, price calculation will undercharge. Either avoid extracting this field for providers/models without a cache_write_mtok, or ensure OpenAI model pricing defines an appropriate cache_write_mtok (often equal to input_mtok if cache writes are billed at the normal input rate).

Copilot uses AI. Check for mistakes.
Comment on lines +5056 to +5058
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change adds support for extracting usage.prompt_tokens_details.cache_write_tokens, but there isn't currently a regression test asserting the extraction behavior (and downstream pricing impact) for an OpenAI-style response that actually includes this field. Please add/extend an existing OpenAI extraction test (e.g. in tests/test_end_to_end.py or tests/test_extract_usage.py) to cover a non-zero cache_write_tokens value.

Copilot uses AI. Check for mistakes.
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -9402,6 +9453,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'audio_tokens'], dest='input_audio_tokens', required=False
),
Expand Down Expand Up @@ -10036,6 +10090,9 @@
UsageExtractorMapping(
path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False
),
UsageExtractorMapping(
path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False
),
Comment on lines 10091 to +10095
Copy link

Copilot AI Mar 30, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

X AI model pricing in this file includes cache_read_mtok but not cache_write_mtok. Because cache_write_tokens are subtracted from uncached input tokens and priced via cache_write_mtok, extracting prompt_tokens_details.cache_write_tokens can undercharge if present and non-zero. Consider removing this mapping for providers/models without cache_write_mtok, or define cache_write_mtok where appropriate.

Copilot uses AI. Check for mistakes.
UsageExtractorMapping(
path=['completion_tokens_details', 'audio_tokens'], dest='output_audio_tokens', required=False
),
Expand Down
Loading