Repository navigation
Conversation
stop_reason_from_str, which the stream accumulator uses to fold a streamed
response into a buffered one, knows content_filter but not refusal, the
spelling Anthropic actually sends. The buffered Anthropic codec does map it
(map_anthropic_stop_reason in codecs/anthropic/buffered.rs), so the same
provider value means two different things depending on whether the response
arrived buffered or was folded out of a stream:
refusal -> Some(Unknown) <- streamed
content_filter -> Some(ContentFilter)
Unknown then encodes back as end_turn for Anthropic and stop for Chat, and
the gen_ai.response.finish_reasons attribute reads unknown, so a refusal is
served to the client as a normal finish.
One arm, plus a test over the whole vocabulary both codecs share.
Signed-off-by: Dmitry Voropaev <dy.voropaev@gmail.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configuration
📒 Files selected for processing (1)
Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review. WalkthroughThe stream parser now maps ChangesStop-reason normalization
Priority: ➖ Normal Estimated code review effort: 2 (Simple) | ~5 minutes Merge Risk: ⚪ Minimal · up to Streamed refusals now retain the same normalized finish reason as buffered responses. No material merge-blocking risk was identified. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
I sniff the stream and find a name, Comment |
stop_reason_from_strincrates/protocol/src/stream.rs:505is what the stream accumulator uses to fold a streamed response into a buffered one. It knowscontent_filterbut notrefusal, which is the spelling Anthropic sends, and its own doc comment says it covers "the common OpenAI and Anthropic spellings".The buffered Anthropic codec does map it, at
codecs/anthropic/buffered.rs:1284:so the same provider value normalizes differently depending on how the response arrived. Folding a single
MessageStopthroughResponseAccumulator:Unknownis not inert from there.anthropic_stop_reasonmaps it to"end_turn"(buffered.rs:1297),openai_finish_reasonto"stop"(openai_chat/buffered.rs:1270), andobservability.rs:169labels itunknown, so thegen_ai.response.finish_reasonsattribute loses it too. A buffered-from-stream Anthropic turn therefore reaches the client as a normal finish withstop_reason: "end_turn", and nothing in the telemetry says a refusal happened.Where that path is taken:
algorithms/util/buffered_response.rs:60buildsBufferedResponse.aggfrom this accumulator, andlibsy-llm-client/src/run.rs:823uses it forremember_canonical_response.One arm. The test covers the whole vocabulary the two codecs share rather than just the new value, since the asymmetry is the actual defect:
On
mainthat test fails onrefusalwithassertion left == right failed: stop reason "refusal"; the other seven pass either way.cargo test --workspace924 passed, 0 failed.cargo fmt --all --checkandcargo clippy -p switchyard-protocol --all-targets -- -D warningsclean on toolchain 1.99.Two notes for review.
#909 edits this same match arm, adding Bedrock's
content_filteredandguardrail_intervenedandmodel_context_window_exceeded, and does not addrefusal. So the two will conflict textually on this line and both are still needed; whichever lands first, the other is a one-line rebase.What this does not cover: a streamed refusal also loses
stop_details, which the buffered codec emits alongsiderefusal. That is a separate hole in the accumulator, which carries no field for it, and it needs a decision about where to keep it rather than a one-line map, so I left it out. Say the word if you want it in the same PR.Summary by CodeRabbit
refusalstop reason is now recognized as a content-filter stop reason.