Add cache_write_tokens mapping to UsageExtractor#330
Add cache_write_tokens mapping to UsageExtractor#330ali-corpo wants to merge 1 commit intopydantic:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Adds extraction support for cache-write token counts from OpenAI-compatible usage.prompt_tokens_details payloads.
Changes:
- Added
UsageExtractorMappingforprompt_tokens_details.cache_write_tokens→cache_write_tokensacross multiple chat-oriented extractors. - Enables downstream pricing/usage calculations to see cache-write token counts when present in provider responses.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False | ||
| ), | ||
| UsageExtractorMapping( | ||
| path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False | ||
| ), |
There was a problem hiding this comment.
cache_write_tokens is treated as a separately-priced bucket in ModelPrice.calc_price() (it is subtracted from uncached input and priced via cache_write_mtok). OpenAI model entries in this data file do not define cache_write_mtok, so if responses start including usage.prompt_tokens_details.cache_write_tokens with a non-zero value, price calculation will undercharge. Either avoid extracting this field for providers/models without a cache_write_mtok, or ensure OpenAI model pricing defines an appropriate cache_write_mtok (often equal to input_mtok if cache writes are billed at the normal input rate).
| UsageExtractorMapping( | ||
| path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False | ||
| ), |
There was a problem hiding this comment.
This change adds support for extracting usage.prompt_tokens_details.cache_write_tokens, but there isn't currently a regression test asserting the extraction behavior (and downstream pricing impact) for an OpenAI-style response that actually includes this field. Please add/extend an existing OpenAI extraction test (e.g. in tests/test_end_to_end.py or tests/test_extract_usage.py) to cover a non-zero cache_write_tokens value.
| path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False | ||
| ), | ||
| UsageExtractorMapping( | ||
| path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False | ||
| ), |
There was a problem hiding this comment.
Deepseek models in this file appear to have cache_read_mtok pricing but no cache_write_mtok. Since cache_write_tokens are priced via cache_write_mtok (and subtracted from uncached input), extracting prompt_tokens_details.cache_write_tokens here can lead to undercharging if the API returns a non-zero value. Consider only mapping this field for providers/models that have a defined cache_write_mtok, or set an explicit cache_write_mtok that matches the provider’s billing semantics.
| path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False | ||
| ), | ||
| UsageExtractorMapping( | ||
| path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False | ||
| ), |
There was a problem hiding this comment.
X AI model pricing in this file includes cache_read_mtok but not cache_write_mtok. Because cache_write_tokens are subtracted from uncached input tokens and priced via cache_write_mtok, extracting prompt_tokens_details.cache_write_tokens can undercharge if present and non-zero. Consider removing this mapping for providers/models without cache_write_mtok, or define cache_write_mtok where appropriate.
| path=['prompt_tokens_details', 'cached_tokens'], dest='cache_read_tokens', required=False | ||
| ), | ||
| UsageExtractorMapping( | ||
| path=['prompt_tokens_details', 'cache_write_tokens'], dest='cache_write_tokens', required=False | ||
| ), |
There was a problem hiding this comment.
Azure’s chat extractor now maps usage.prompt_tokens_details.cache_write_tokens into cache_write_tokens, but Azure/OpenAI-priced models in this file don’t define cache_write_mtok. Since cache_write_tokens are subtracted from uncached input and priced via cache_write_mtok, a non-zero value from the API would undercharge. Consider only extracting this field when a cache_write_mtok is defined (or ensure the pricing model includes the appropriate cache-write rate).
Add cache_write_tokens mapping to UsageExtractor