Skip to content

fix(protocol): normalize Anthropic's refusal stop reason when streaming - #941

Open
v0ropaev wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
v0ropaev:fix/streamed-refusal-stop-reason
Open

v0ropaev wants to merge 1 commit into
NVIDIA-NeMo:mainfrom
v0ropaev:fix/streamed-refusal-stop-reason

Conversation

@v0ropaev

@v0ropaev v0ropaev commented Oct 7, 2026 •

Copy link
Copy Markdown

stop_reason_from_str in crates/protocol/src/stream.rs:505 is what the stream accumulator uses to fold a streamed response into a buffered one. It knows content_filter but not refusal, which is the spelling Anthropic sends, and its own doc comment says it covers "the common OpenAI and Anthropic spellings".

The buffered Anthropic codec does map it, at codecs/anthropic/buffered.rs:1284:

Some("refusal") => StopReason::ContentFilter,

so the same provider value normalizes differently depending on how the response arrived. Folding a single MessageStop through ResponseAccumulator:

refusal        -> Some(Unknown)
content_filter -> Some(ContentFilter)

Unknown is not inert from there. anthropic_stop_reason maps it to "end_turn" (buffered.rs:1297), openai_finish_reason to "stop" (openai_chat/buffered.rs:1270), and observability.rs:169 labels it unknown, so the gen_ai.response.finish_reasons attribute loses it too. A buffered-from-stream Anthropic turn therefore reaches the client as a normal finish with stop_reason: "end_turn", and nothing in the telemetry says a refusal happened.

Where that path is taken: algorithms/util/buffered_response.rs:60 builds BufferedResponse.agg from this accumulator, and libsy-llm-client/src/run.rs:823 uses it for remember_canonical_response.

One arm. The test covers the whole vocabulary the two codecs share rather than just the new value, since the asymmetry is the actual defect:

length, max_tokens      -> MaxTokens
tool_calls, tool_use    -> ToolUse
content_filter, refusal -> ContentFilter
stop, end_turn          -> EndTurn

On main that test fails on refusal with assertion left == right failed: stop reason "refusal"; the other seven pass either way.

cargo test --workspace 924 passed, 0 failed. cargo fmt --all --check and cargo clippy -p switchyard-protocol --all-targets -- -D warnings clean on toolchain 1.99.

Two notes for review.

#909 edits this same match arm, adding Bedrock's content_filtered and guardrail_intervened and model_context_window_exceeded, and does not add refusal. So the two will conflict textually on this line and both are still needed; whichever lands first, the other is a one-line rebase.

What this does not cover: a streamed refusal also loses stop_details, which the buffered codec emits alongside refusal. That is a separate hole in the accumulator, which carries no field for it, and it needs a decision about where to keep it rather than a one-line map, so I left it out. Say the word if you want it in the same PR.

Summary by CodeRabbit

  • Bug Fixes
    • Anthropic’s refusal stop reason is now recognized as a content-filter stop reason.

stop_reason_from_str, which the stream accumulator uses to fold a streamed
response into a buffered one, knows content_filter but not refusal, the
spelling Anthropic actually sends. The buffered Anthropic codec does map it
(map_anthropic_stop_reason in codecs/anthropic/buffered.rs), so the same
provider value means two different things depending on whether the response
arrived buffered or was folded out of a stream:

    refusal        -> Some(Unknown)          <- streamed
    content_filter -> Some(ContentFilter)

Unknown then encodes back as end_turn for Anthropic and stop for Chat, and
the gen_ai.response.finish_reasons attribute reads unknown, so a refusal is
served to the client as a normal finish.

One arm, plus a test over the whole vocabulary both codecs share.

Signed-off-by: Dmitry Voropaev <dy.voropaev@gmail.com>
@v0ropaev
v0ropaev requested a review from a team as a code owner October 7, 2026 16:02
@coderabbitai

coderabbitai Bot commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Repository: NVIDIA-NeMo/Switchyard/.coderabbit.yaml
  • Review profile: CHILL
  • Plan: Enterprise
  • Run ID: 1ebadb96-8b2e-4031-8f47-9166a590e3d9
📥 Commits

Reviewing files that changed from the base of the PR and between a3cdc51 and 1c9c8ae.

📒 Files selected for processing (1)
  • crates/protocol/src/stream.rs

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.


Walkthrough

The stream parser now maps refusal to StopReason::ContentFilter, alongside content_filter. A test checks normalization for the listed stop-reason spellings.

Changes

Stop-reason normalization

Layer / File(s) Summary
Map and test stop-reason spellings
crates/protocol/src/stream.rs
stop_reason_from_str maps refusal to StopReason::ContentFilter. A test checks normalization for the listed stop-reason spellings.

Priority: ➖ Normal

Estimated code review effort: 2 (Simple) | ~5 minutes

Merge Risk: ⚪ Minimal · up to 1c9c8

Streamed refusals now retain the same normalized finish reason as buffered responses. No material merge-blocking risk was identified.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: normalizing Anthropic's streamed refusal stop reason in the protocol.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 4 functions across 1 files.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

I sniff the stream and find a name,
“refusal” joins the filtered frame.
The tests now check each spelling right,
I hop through values, neat and bright.
Then tuck my ears and end the night.

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant