Skip to content

fix: 12980 align nativeStructuredOuptut across bedrock and anthropic #12981

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
heiwen wants to merge 2 commits into from
+174 −38
Closed

fix: 12980 align nativeStructuredOuptut across bedrock and anthropic #12981

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1b1907e
fix: 12980 align nativeStructuredOuptut across bedrock and anthropic
heiwen Mar 2, 2026
effd107
docs: add changeset
heiwen Apr 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/curly-crabs-juggle.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@ai-sdk/amazon-bedrock': patch
---

fix(bedrock): use native structured outputs for supported Anthropic models
195 changes: 158 additions & 37 deletions packages/amazon-bedrock/src/bedrock-chat-language-model.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,12 @@ const fakeFetchWithAuth = injectFetchHeaders({ 'x-amz-auth': 'test-auth' });

const modelId = 'anthropic.claude-3-haiku-20240307-v1:0';
const anthropicModelId = 'anthropic.claude-3-5-sonnet-20240620-v1:0'; // Define at top level
const supportedStructuredOutputModelId =
'anthropic.claude-sonnet-4-5-20250929-v1:0';
const supportedStructuredOutput41ModelId =
'anthropic.claude-opus-4-1-20250805-v1:0';
const dateBasedStructuredOutput4ModelId =
'anthropic.claude-sonnet-4-20250514-v1:0';
const baseUrl = 'https://bedrock-runtime.us-east-1.amazonaws.com';

const streamUrl = `${baseUrl}/model/${encodeURIComponent(
Expand All @@ -77,6 +83,15 @@ const generateUrl = `${baseUrl}/model/${encodeURIComponent(modelId)}/converse`;
const anthropicGenerateUrl = `${baseUrl}/model/${encodeURIComponent(
anthropicModelId,
)}/converse`;
const supportedStructuredOutputGenerateUrl = `${baseUrl}/model/${encodeURIComponent(
supportedStructuredOutputModelId,
)}/converse`;
const supportedStructuredOutput41GenerateUrl = `${baseUrl}/model/${encodeURIComponent(
supportedStructuredOutput41ModelId,
)}/converse`;
const dateBasedStructuredOutput4GenerateUrl = `${baseUrl}/model/${encodeURIComponent(
dateBasedStructuredOutput4ModelId,
)}/converse`;
Comment on lines +86 to +94
Copy link
Copy Markdown
Collaborator

@aayush-kapoor aayush-kapoor Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure why we have to create these 3 vars globally instead of locally in the tests


const novaModelId = 'us.amazon.nova-2-lite-v1:0';
const novaGenerateUrl = `${baseUrl}/model/${encodeURIComponent(
Expand All @@ -103,16 +118,22 @@ const server = createTestServer({
},
// Configure the server for the Anthropic model from the start
[anthropicGenerateUrl]: {},
[supportedStructuredOutputGenerateUrl]: {},
[supportedStructuredOutput41GenerateUrl]: {},
[dateBasedStructuredOutput4GenerateUrl]: {},
[novaGenerateUrl]: {},
[openaiGenerateUrl]: {},
[newerAnthropicGenerateUrl]: {},
});

function prepareJsonFixtureResponse(
filename: string,
{ headers }: { headers?: Record<string, string> } = {},
{
headers,
url = generateUrl,
}: { headers?: Record<string, string>; url?: string } = {},
) {
server.urls[generateUrl].response = {
server.urls[url].response = {
type: 'json-value',
headers,
body: JSON.parse(
Expand Down Expand Up @@ -149,6 +170,18 @@ beforeEach(() => {
type: 'json-value',
body: {},
};
server.urls[supportedStructuredOutputGenerateUrl].response = {
type: 'json-value',
body: {},
};
server.urls[supportedStructuredOutput41GenerateUrl].response = {
type: 'json-value',
body: {},
};
server.urls[dateBasedStructuredOutput4GenerateUrl].response = {
type: 'json-value',
body: {},
};
mockPrepareAnthropicTools.mockClear();
});

Expand Down Expand Up @@ -183,6 +216,36 @@ const newerAnthropicModel = new BedrockChatLanguageModel(
},
);

const supportedStructuredOutputModel = new BedrockChatLanguageModel(
supportedStructuredOutputModelId,
{
baseUrl: () => baseUrl,
headers: {},
fetch: fakeFetchWithAuth,
generateId: () => 'test-id',
},
);

const supportedStructuredOutput41Model = new BedrockChatLanguageModel(
supportedStructuredOutput41ModelId,
{
baseUrl: () => baseUrl,
headers: {},
fetch: fakeFetchWithAuth,
generateId: () => 'test-id',
},
);

const dateBasedStructuredOutput4Model = new BedrockChatLanguageModel(
dateBasedStructuredOutput4ModelId,
{
baseUrl: () => baseUrl,
headers: {},
fetch: fakeFetchWithAuth,
generateId: () => 'test-id',
},
);

let mockOptions: { success: boolean; errorValue?: any } = { success: true };

describe('doStream', () => {
Expand Down Expand Up @@ -4403,9 +4466,11 @@ describe('doGenerate', () => {
});

it('should use native output_config.format instead of json tool when thinking is enabled with structured output', async () => {
prepareJsonFixtureResponse('bedrock-text');
prepareJsonFixtureResponse('bedrock-text', {
url: supportedStructuredOutputGenerateUrl,
});

await model.doGenerate({
await supportedStructuredOutputModel.doGenerate({
prompt: [
{
role: 'user',
Expand Down Expand Up @@ -4467,9 +4532,11 @@ describe('doGenerate', () => {
});

it('should merge output_config.effort and output_config.format when thinking with maxReasoningEffort and structured output', async () => {
prepareJsonFixtureResponse('bedrock-text');
prepareJsonFixtureResponse('bedrock-text', {
url: supportedStructuredOutputGenerateUrl,
});

await model.doGenerate({
await supportedStructuredOutputModel.doGenerate({
prompt: [
{
role: 'user',
Expand Down Expand Up @@ -4516,30 +4583,95 @@ describe('doGenerate', () => {
});
});

it('should still use json tool fallback for structured output without thinking enabled', async () => {
server.urls[generateUrl].response = {
type: 'json-value',
body: {
output: {
message: {
role: 'assistant',
content: [
{
toolUse: {
toolUseId: 'json-tool-id',
name: 'json',
input: { name: 'Test' },
},
},
],
it('should use native output_config.format for supported Anthropic models without thinking enabled', async () => {
prepareJsonFixtureResponse('bedrock-text', {
url: supportedStructuredOutputGenerateUrl,
});

await supportedStructuredOutputModel.doGenerate({
prompt: [
{
role: 'user',
content: [{ type: 'text', text: 'Generate a name' }],
},
],
responseFormat: {
type: 'json',
schema: {
type: 'object',
properties: {
name: { type: 'string' },
},
required: ['name'],
},
usage: { inputTokens: 4, outputTokens: 10, totalTokens: 14 },
stopReason: 'tool_use',
},
};
});

const result = await model.doGenerate({
const requestBody = await server.calls[0].requestBodyJson;

expect(requestBody.toolConfig).toBeUndefined();

expect(requestBody.additionalModelRequestFields?.output_config).toEqual({
format: {
type: 'json_schema',
schema: {
type: 'object',
properties: {
name: { type: 'string' },
},
required: ['name'],
},
},
});
});

it('should use native output_config.format for Anthropic 4-1 models without thinking enabled', async () => {
prepareJsonFixtureResponse('bedrock-text', {
url: supportedStructuredOutput41GenerateUrl,
});

await supportedStructuredOutput41Model.doGenerate({
prompt: [
{
role: 'user',
content: [{ type: 'text', text: 'Generate a name' }],
},
],
responseFormat: {
type: 'json',
schema: {
type: 'object',
properties: {
name: { type: 'string' },
},
required: ['name'],
},
},
});

const requestBody = await server.calls[0].requestBodyJson;

expect(requestBody.toolConfig).toBeUndefined();
expect(requestBody.additionalModelRequestFields?.output_config).toEqual({
format: {
type: 'json_schema',
schema: {
type: 'object',
properties: {
name: { type: 'string' },
},
required: ['name'],
},
},
});
});

it('should use json tool fallback for date-based Anthropic 4 models without thinking enabled', async () => {
prepareJsonFixtureResponse('bedrock-text', {
url: dateBasedStructuredOutput4GenerateUrl,
});

await dateBasedStructuredOutput4Model.doGenerate({
prompt: [
{
role: 'user',
Expand All @@ -4564,20 +4696,9 @@ describe('doGenerate', () => {
expect(requestBody.toolConfig.tools).toHaveLength(1);
expect(requestBody.toolConfig.tools[0].toolSpec.name).toBe('json');
expect(requestBody.toolConfig.toolChoice).toEqual({ any: {} });

expect(
requestBody.additionalModelRequestFields?.output_config?.format,
).toBeUndefined();

expect(result.content).toMatchInlineSnapshot(`
[
{
"text": "{"name":"Test"}",
"type": "text",
},
]
`);
expect(result.providerMetadata?.bedrock?.isJsonResponseFromTool).toBe(true);
});

it('should extract reasoning text with signature', async () => {
Expand Down
12 changes: 11 additions & 1 deletion packages/amazon-bedrock/src/bedrock-chat-language-model.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ export class BedrockChatLanguageModel implements LanguageModelV4 {

const useNativeStructuredOutput =
isAnthropicModel &&
isThinkingEnabled &&
supportsAnthropicNativeStructuredOutput(this.modelId) &&
responseFormat?.type === 'json' &&
responseFormat.schema != null;

Expand Down Expand Up @@ -987,6 +987,16 @@ export class BedrockChatLanguageModel implements LanguageModelV4 {
}
}

function supportsAnthropicNativeStructuredOutput(modelId: string): boolean {
// only version 4+ supports structured output
const v = modelId.indexOf('-4-');
if (v < 0) return false;

// versions before 4-1 don't suport structured outputs yet
// reject bedrock style claude-opus-4-20250514-v1:0
return modelId[v + 3] !== '-' && modelId[v + 4] === '-';
}

Comment on lines +990 to +999
Copy link
Copy Markdown
Collaborator

@aayush-kapoor aayush-kapoor Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

imo this is very brittle logic. we would keep coming back to this whenever newer models are added

const BedrockStopReasonSchema = z.union([
z.enum(BEDROCK_STOP_REASONS),
z.string(),
Expand Down
Loading