Add count-array extraction for counting tool calls in response arrays#290
Add count-array extraction for counting tool calls in response arrays#290Kludex wants to merge 1 commit intoadd-web-search-request-pricingfrom
Conversation
Adds a `count` field to `UsageExtractorMapping` that navigates from the response root to an array and counts items matching a condition. This enables extracting web_search and file_search call counts from OpenAI's Responses API output array, where tool calls appear as typed items rather than in the usage object.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| for (const step of steps) { | ||
| if (typeof step === 'string') { | ||
| if (mappingCheck.guard(data)) { | ||
| data = data[step] | ||
| } else if (required) { | ||
| throw new Error(`Expected mapping at count path, got ${typeName(data)}`) | ||
| } else { | ||
| return null | ||
| } | ||
| } |
There was a problem hiding this comment.
🟡 JS countArrayItems silently ignores non-string (ArrayMatch) path steps
The countArrayItems function accepts count.path of type ExtractPath, which can contain ArrayMatch objects. However, the loop at lines 207-223 only navigates when typeof step === 'string' (line 208). When a step is an ArrayMatch object, the entire navigation for that step is skipped, but the undefined check at line 217 still runs against the unchanged data.
Root Cause and Impact
The if (typeof step === 'string') block (line 208) handles string steps, but there is no else branch handling ArrayMatch steps. The undefined check at line 217 is outside the string-check block, so it runs for every step — but when the step is an ArrayMatch, data was never modified, so it still holds the previous value.
For example, if count.path were ["output", {field: "type", match: ..., type: "array-match"}], the ArrayMatch step would be silently skipped, and the function would try to count items in the wrong data (the parent array instead of a matched sub-object).
The Python _count_array_items at packages/python/genai_prices/types.py:562-574 has the identical issue — only isinstance(step, str) is handled.
Currently this doesn't affect production because the YAML config only uses path: output (a simple string), but the type contract (ExtractPath) permits ArrayMatch in count paths, and the function will silently produce incorrect results for such inputs rather than raising an error.
Prompt for agents
In packages/js/src/extractUsage.ts, the countArrayItems function (lines 207-223) only handles string steps from the path but silently ignores ArrayMatch steps. Either add an else branch at line 216 that throws an error like `throw new Error('ArrayMatch steps are not supported in count paths')` when the step is not a string, or add proper handling for ArrayMatch objects similar to the extractPath function. The same fix should be applied to the Python _count_array_items function in packages/python/genai_prices/types.py at lines 562-574, adding handling for ArrayMatch steps (or raising ValueError for unsupported step types).
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Summary
countfield toUsageExtractorMappingthat navigates from the response root to an array and counts items matching a conditionpathoptional onUsageExtractorMapping- exactly one ofpathorcountmust be setTest plan