Normalize OpenAI token params and expose provider finish reasons#709
Normalize OpenAI token params and expose provider finish reasons#709losingle wants to merge 5 commits intocrmne:mainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves cross-provider response normalization in RubyLLM by (1) making OpenAI chat-completions token parameters compatible with newer OpenAI-style endpoints and (2) propagating a normalized finish_reason through both sync and streaming flows (including stream accumulation).
Changes:
- OpenAI provider now mirrors
max_tokensintomax_completion_tokenswhen the latter is not explicitly provided. - Added
finish_reasontoRubyLLM::Message/Chunk, normalized provider-specific stop reasons (OpenAI/Anthropic/Gemini), and preserved the final non-nil finish reason inStreamAccumulator. - Added/updated focused RSpec coverage across providers and streaming paths; cleaned up spec lint/quoting issues.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| spec/ruby_llm/stream_accumulator_spec.rb | Adds coverage that the accumulator keeps the last non-nil finish_reason. |
| spec/ruby_llm/providers/open_ai/streaming_spec.rb | Verifies OpenAI streaming chunks expose normalized finish_reason. |
| spec/ruby_llm/providers/open_ai/chat_spec.rb | Verifies OpenAI sync message parsing normalizes finish_reason (e.g., tool_calls → tool_use). |
| spec/ruby_llm/providers/open_ai_provider_spec.rb | Verifies OpenAI provider param normalization for max_tokens/max_completion_tokens. |
| spec/ruby_llm/providers/gemini/streaming_spec.rb | Verifies Gemini streaming chunk finish_reason normalization (e.g., MAX_TOKENS → length). |
| spec/ruby_llm/providers/gemini/chat_spec.rb | Verifies Gemini sync message finish_reason normalization (e.g., SAFETY → content_filter). |
| spec/ruby_llm/providers/anthropic/streaming_spec.rb | Verifies Anthropic streaming chunk finish_reason normalization (e.g., max_tokens → length). |
| spec/ruby_llm/providers/anthropic/chat_spec.rb | Verifies Anthropic sync message finish_reason normalization. |
| spec/ruby_llm/generators/chat_ui_generator_spec.rb | Adjusts string expectations to resolve spec lint/quoting issues. |
| lib/ruby_llm/stream_accumulator.rb | Tracks and emits the final non-nil finish_reason when assembling a message from chunks. |
| lib/ruby_llm/providers/openai/streaming.rb | Adds normalized finish_reason to OpenAI streaming chunks. |
| lib/ruby_llm/providers/openai/chat.rb | Adds normalized finish_reason to OpenAI parsed messages + normalization helper. |
| lib/ruby_llm/providers/openai.rb | Adds OpenAI-specific request param normalization for max_completion_tokens. |
| lib/ruby_llm/providers/gemini/streaming.rb | Adds normalized finish_reason to Gemini streaming chunks + mapping helper. |
| lib/ruby_llm/providers/gemini/chat.rb | Adds normalized finish_reason to Gemini parsed messages + mapping helper. |
| lib/ruby_llm/providers/anthropic/streaming.rb | Adds normalized finish_reason to Anthropic streaming chunks + mapping helper. |
| lib/ruby_llm/providers/anthropic/chat.rb | Adds normalized finish_reason to Anthropic parsed messages + mapping helper. |
| lib/ruby_llm/message.rb | Adds finish_reason to Message API and serialization (to_h). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| def normalize_finish_reason(reason) | ||
| case reason | ||
| when 'STOP' | ||
| 'stop' | ||
| when 'MAX_TOKENS' | ||
| 'length' | ||
| when 'SAFETY', 'RECITATION' | ||
| 'content_filter' | ||
| else | ||
| reason | ||
| end |
There was a problem hiding this comment.
normalize_finish_reason duplicates the same mapping logic that also exists in the Gemini streaming implementation. Consider extracting the mapping into a shared helper/constant (e.g., a single module method used by both Chat and Streaming) to prevent the two paths from diverging over time.
| def normalize_finish_reason(reason) | |
| case reason | |
| when 'STOP' | |
| 'stop' | |
| when 'MAX_TOKENS' | |
| 'length' | |
| when 'SAFETY', 'RECITATION' | |
| 'content_filter' | |
| else | |
| reason | |
| end | |
| FINISH_REASON_MAP = { | |
| 'STOP' => 'stop', | |
| 'MAX_TOKENS' => 'length', | |
| 'SAFETY' => 'content_filter', | |
| 'RECITATION' => 'content_filter' | |
| }.freeze | |
| def normalize_finish_reason(reason) | |
| FINISH_REASON_MAP.fetch(reason, reason) |
| def normalize_finish_reason(reason) | ||
| case reason | ||
| when 'end_turn', 'stop_sequence' | ||
| 'stop' | ||
| when 'max_tokens' | ||
| 'length' | ||
| else | ||
| reason | ||
| end |
There was a problem hiding this comment.
normalize_finish_reason is duplicated between Anthropic chat and streaming code paths. Consider centralizing the mapping (shared helper/constant) so any future stop_reason additions/changes stay consistent across sync vs streaming parsing.
| def normalize_finish_reason(reason) | |
| case reason | |
| when 'end_turn', 'stop_sequence' | |
| 'stop' | |
| when 'max_tokens' | |
| 'length' | |
| else | |
| reason | |
| end | |
| FINISH_REASON_MAP = { | |
| 'end_turn' => 'stop', | |
| 'stop_sequence' => 'stop', | |
| 'max_tokens' => 'length' | |
| }.freeze | |
| def normalize_finish_reason(reason) | |
| FINISH_REASON_MAP.fetch(reason, reason) |
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
This PR improves provider normalization in two areas:
OpenAI chat-completions compatibility
When max_tokens is provided and max_completion_tokens is not, the OpenAI provider now mirrors the value to max_completion_tokens. This keeps existing callers working while improving compatibility with newer OpenAI-compatible endpoints that expect max_completion_tokens.
Finish reason propagation
Assistant messages and streaming chunks now expose a normalized finish_reason across OpenAI, Anthropic, and Gemini. The stream accumulator also preserves the final non-nil finish reason on the assembled message.
What changed
Validation