feat(ai/core): lazy tool discovery for token-efficient large tool sets

[fkesheh]

Summary

Implements lazy tool discovery as requested in #13423. Tools marked with lazy: true send only their name and description to the LLM — the full inputSchema is omitted. A synthetic __load_tool_schema__ tool is auto-injected so the LLM can fetch full schemas on demand before calling a tool.

How it works:

1. Lazy tools are sent to the LLM with a minimal empty schema ({ type: 'object', properties: {} }) instead of their full inputSchema
2. The LLM calls __load_tool_schema__ to get the full schema as a tool result (text)
3. The LLM calls the lazy tool with arguments based on the loaded schema
4. If the LLM sends strings for nested objects/arrays (common with empty schemas), the SDK auto-repairs by JSON.parsing those values against the real schema, then validates and executes

This reduces token usage and improves tool selection accuracy for apps with large tool sets (20-50+ tools).

Prior art: TanStack AI lazy tool discovery, Cursor dynamic context discovery (46.9% token reduction in A/B test).

Design principles

• Stateless — no tracking of what's been loaded, no message scanning
• Cache-friendly — same tool list every step, preserves LLM prompt cache
• All tools always visible (name + description) and always executable
• Overridable — users can define their own __load_tool_schema__ tool
• skill field — optional extended docs loaded on demand for lazy tools

Key implementation details

• prepareTools: lazy tools emit minimal empty schema with instruction to load schema first
• parseToolCall: for lazy tools, auto-repairs string→object/array mismatches using the real schema
• __load_tool_schema__: returns serialized JSON Schema (stripped of Zod internals via JSON.parse/stringify)
• buildEffectiveTools: shared helper used by both generateText and streamText
• filterActiveTools: preserves __load_tool_schema__ when activeTools filtering is used

Changes

• @ai-sdk/provider-utils: Add lazy?: boolean and skill?: string to Tool type
• ai: New load-tool-schema.ts — synthetic tool factory + buildEffectiveTools helper
• ai: prepareTools emits minimal empty schema for lazy tools
• ai: parseToolCall auto-repairs lazy tool inputs (JSON.parse strings that should be objects/arrays)
• ai: filterActiveTools preserves __load_tool_schema__ through activeTools filtering
• ai: Auto-inject __load_tool_schema__ when lazy tools exist (overridable)

Test plan

• Unit tests for createLoadToolSchemaTool (schema return, skill, inputExamples, error entries)
• Unit tests for buildEffectiveTools (no-op without lazy, injection, user override)
• Unit tests for prepareTools (minimal schema for lazy, mixed lazy/eager)
• All 2210 existing tests pass
• Full monorepo build (69/69 tasks)
• Lint/format clean
• Tested in real app (legbi) with 33 tools, 17 lazy

Closes #13423

🤖 Generated with Claude Code

[ziyadev]

If the LLM calls __load_tool_schema__({ toolNames: [] }), the execute handler returns {} silently, and the LLM may loop calling it again with no progress. The fix could be simple by just return a warning message for empty input and explicitly tell the LLM in the tool's description to always pass at least one tool name.

[fkesheh]

very good catch, I will add that

[vercel]

Additional Suggestion:

filterActiveTools strips the auto-injected __load_tool_schema__ tool when user specifies activeTools, silently breaking lazy tool discovery.

[fkesheh]

Alternative approach: modelInputSchema for user-land discovery harness

After implementing the baked-in approach above, I explored a lighter alternative that gives users full control without adding SDK-specific discovery logic.

The gap

Users can already:

• Add synthetic tools to the tools map (their own discovery tool)
• Use prepareStep to change activeTools, toolChoice, model, messages per step
• Use toolCallRepair to fix malformed tool calls

The one missing piece: users can't send a different schema to the LLM than the one used for validation. prepareTools always uses inputSchema for both.

Proposed: modelInputSchema

Add an optional modelInputSchema field to the Tool type. When present, prepareTools sends it to the LLM instead of inputSchema. Validation still uses inputSchema.

const myLazyTool = tool({
  description: 'Create entity type',
  inputSchema: z.object({ slug: z.string(), labels: z.object({...}), fields: z.array(...) }),
  modelInputSchema: z.object({}), // minimal schema sent to LLM
  execute: async (input) => { ... },
});

const discoverTools = tool({
  description: 'Get schema for tools',
  inputSchema: z.object({ toolNames: z.array(z.string()) }),
  execute: async ({ toolNames }) => { /* return schemas */ },
});

streamText({
  tools: { create_entity_type: myLazyTool, discover_tools: discoverTools },
  toolCallRepair: async ({ toolCall, tools, error }) => {
    // user-land: repair string→object mismatches for lazy tools
  },
});

Trade-offs

Aspect	Current PR (baked-in)	modelInputSchema alternative
SDK changes	~8 files, new synthetic tool, auto-repair	2 files: Tool type + prepareTools
User effort	Zero — just add lazy: true	User builds discovery tool + repair
Flexibility	Opinionated flow	Users own the entire harness
Maintenance	More SDK surface to maintain	Minimal SDK surface

Both approaches could coexist — modelInputSchema as the primitive, lazy: true as the batteries-included convenience built on top.

Happy to implement modelInputSchema as a separate PR if the team prefers the composable-primitives approach.