Skip to content

fix: price OpenAI requests by actual service tier - #4625

Merged
aheritier merged 1 commit into
docker:mainfrom
dgageot:worktree-board-9568ac1ee9f491c0
Oct 17, 2026
Merged

aheritier merged 1 commit into
docker:mainfrom
dgageot:worktree-board-9568ac1ee9f491c0

Conversation

@dgageot

@dgageot dgageot commented Oct 17, 2026

Copy link
Copy Markdown
Member

Why

OpenAI's fast/priority/ultrafast service tiers charge a premium over
the base model rate, but that premium wasn't tracked per-model: it was a
hardcoded list of model names that would silently stop applying once a model
rolled off the allowlist, and never covered models with a different premium
than 2x/6x.

What

  • Replaced the model-name allowlist in Cost.ForServiceTier with a
    fastMultiplier derived per-model from the catalogue (withFastPricing in
    pkg/modelsdev/service_tier.go), looked up once in Store.GetModel.
  • The multiplier is only trusted when the catalogue's base rates match our
    own and the premium is uniform across input/output/cache rates and all
    context tiers; otherwise it falls back to the previous flat estimates
    (fast = 2x, ultrafast = 6x), as before.
  • Dated model snapshots (e.g. -2025-01-01 suffix) still fall back to the
    alias's premium, but no longer assume the alias's base rates apply to them.
  • Endpoint/override/cache/context pricing behavior is unchanged.
  • Updated docs and the examples/openai-service-tier.yaml example, and
    extended tests in pkg/modelsdev and pkg/runtime.

Validation

  • CI=true task build, task test, task lint
  • Targeted -race runs for the modelsdev/runtime service-tier tests and
    the catalogue boundary tests
  • Manual smoke test against live gpt-6.1-sol ultrafast pricing

Remove the model allowlist and instead derive the Fast tier premium
from matching catalogue entries, falling back to the documented
multiplier when metadata is missing or inconsistent. Endpoint and
cost-override guards, tests, and docs are preserved.

Signed-off-by: David Gageot <david.gageot@docker.com>
@dgageot
dgageot requested a review from a team as a code owner October 17, 2026 11:31
@aheritier aheritier added area/docs Documentation changes area/models LLM model integrations and model providers area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/fix PR fixes a bug (maps to fix:). Use on PRs only. labels Oct 17, 2026
@aheritier
aheritier added this pull request to the merge queue Oct 17, 2026
Merged via the queue into docker:main with commit b8348bf Oct 17, 2026
22 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation changes area/models LLM model integrations and model providers area/providers/openai For features/issues/fixes related to the usage of OpenAI models area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection kind/fix PR fixes a bug (maps to fix:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants