feat(inference): add native Anthropic (Claude) provider#2890
feat(inference): add native Anthropic (Claude) provider#2890johnford2002 wants to merge 6 commits into
Conversation
Includes the ANTHROPIC_* config schema/fields so the new client's fromConfig() typechecks under the pre-commit gate.
Greptile SummaryThis PR adds a first-class Anthropic (Claude) inference provider to Karakeep, selected when
Confidence Score: 3/5The additive change is well-scoped, but the Anthropic client silently degrades to plain-text output in json mode when no schema is given — a behavioural difference from the OpenAI path that could cause parse failures for any existing deployment that switches providers with INFERENCE_OUTPUT_SCHEMA=json. The buildAnthropicOutputConfig function returns undefined (no JSON enforcement) when outputSchema === json and schema === null, whereas OpenAI always sends { type: json_object } in that case. Any caller relying on json-mode JSON output without an explicit schema will silently receive unstructured text from Anthropic and likely break at the parsing step. The rest of the change is correct. packages/shared/inference.ts — specifically the buildAnthropicOutputConfig helper and the absence of timeout wiring in the AnthropicInferenceClient constructor Important Files Changed
Prompt To Fix All With AIFix the following 4 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 4
packages/shared/inference.ts:191-205
**Silent no-op in `json` mode without a schema**
`buildAnthropicOutputConfig` returns `undefined` when `outputSchema === "json"` and `schema === null`. This silently drops JSON enforcement: Anthropic gets a plain-text request instead. The OpenAI path always sends `{ type: "json_object" }` in `json` mode regardless of whether a schema is supplied, so any caller that sets `INFERENCE_OUTPUT_SCHEMA=json` without a schema (which is valid OpenAI usage) will receive unstructured text from Anthropic and likely fail to parse it. At minimum a warning should be logged, or the condition should be tightened to make the gap explicit.
### Issue 2 of 4
packages/shared/inference.ts:393-399
**No configurable timeout for the Anthropic client**
`OpenAIInferenceClient` reads `OPENAI_TIMEOUT_SEC` and passes it as `timeout` to the `OpenAI` constructor. The `AnthropicInferenceClient` passes no timeout to `new Anthropic({...})`, so the SDK's built-in default (~10 minutes) is always used, ignoring any user-configured timeout expectation. Consider adding an `ANTHROPIC_TIMEOUT_SEC` env var (or re-using a generic `INFERENCE_TIMEOUT_SEC`) and wiring it through `AnthropicInferenceConfig`.
### Issue 3 of 4
packages/shared/inference.ts:179-187
**Ambiguous log message: text vs. image model substitution indistinguishable**
`resolveAnthropicModel` is called for both `textModel` and `imageModel`, but the log message always says "No Claude model set … Set INFERENCE_TEXT_MODEL/INFERENCE_IMAGE_MODEL to override." — it never identifies which model slot triggered the substitution. When both are defaulted, users see the same message twice with no way to tell them apart. Passing a `slot` identifier (e.g. `"text"` / `"image"`) would make the message actionable.
### Issue 4 of 4
packages/shared/inference.test.ts:17-25
**Test fixture uses OpenAI model names for an Anthropic client under test**
`makeClient()` defaults `textModel: "gpt-4.1-mini"` and `imageModel: "gpt-4o-mini"`. The tests exercise the substitution path (good), but the helper's name and intent could mislead a future contributor into thinking it simulates a misconfigured OpenAI client rather than an Anthropic client with OpenAI-defaulted models. A brief comment explaining why the OpenAI defaults are intentional here would improve clarity.
Reviews (1): Last reviewed commit: "docs: document the native Anthropic infe..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| function buildAnthropicOutputConfig( | ||
| // eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
| schema: z.ZodSchema<any> | null, | ||
| outputSchema: "structured" | "json" | "plain", | ||
| ) { | ||
| if (!schema || outputSchema === "plain") { | ||
| return undefined; | ||
| } | ||
| return { | ||
| format: { | ||
| type: "json_schema" as const, | ||
| schema: z.toJSONSchema(schema), | ||
| }, | ||
| }; | ||
| } |
There was a problem hiding this comment.
json mode without a schema
buildAnthropicOutputConfig returns undefined when outputSchema === "json" and schema === null. This silently drops JSON enforcement: Anthropic gets a plain-text request instead. The OpenAI path always sends { type: "json_object" } in json mode regardless of whether a schema is supplied, so any caller that sets INFERENCE_OUTPUT_SCHEMA=json without a schema (which is valid OpenAI usage) will receive unstructured text from Anthropic and likely fail to parse it. At minimum a warning should be logged, or the condition should be tightened to make the gap explicit.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/shared/inference.ts
Line: 191-205
Comment:
**Silent no-op in `json` mode without a schema**
`buildAnthropicOutputConfig` returns `undefined` when `outputSchema === "json"` and `schema === null`. This silently drops JSON enforcement: Anthropic gets a plain-text request instead. The OpenAI path always sends `{ type: "json_object" }` in `json` mode regardless of whether a schema is supplied, so any caller that sets `INFERENCE_OUTPUT_SCHEMA=json` without a schema (which is valid OpenAI usage) will receive unstructured text from Anthropic and likely fail to parse it. At minimum a warning should be logged, or the condition should be tightened to make the gap explicit.
How can I resolve this? If you propose a fix, please make it concise.| OPENAI_DEFAULT_IMAGE_MODEL, | ||
| ); | ||
| this.anthropic = new Anthropic({ | ||
| apiKey: config.apiKey, | ||
| baseURL: config.baseURL, | ||
| }); | ||
| } |
There was a problem hiding this comment.
OpenAIInferenceClient reads OPENAI_TIMEOUT_SEC and passes it as timeout to the OpenAI constructor. The AnthropicInferenceClient passes no timeout to new Anthropic({...}), so the SDK's built-in default (~10 minutes) is always used, ignoring any user-configured timeout expectation. Consider adding an ANTHROPIC_TIMEOUT_SEC env var (or re-using a generic INFERENCE_TIMEOUT_SEC) and wiring it through AnthropicInferenceConfig.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/shared/inference.ts
Line: 393-399
Comment:
**No configurable timeout for the Anthropic client**
`OpenAIInferenceClient` reads `OPENAI_TIMEOUT_SEC` and passes it as `timeout` to the `OpenAI` constructor. The `AnthropicInferenceClient` passes no timeout to `new Anthropic({...})`, so the SDK's built-in default (~10 minutes) is always used, ignoring any user-configured timeout expectation. Consider adding an `ANTHROPIC_TIMEOUT_SEC` env var (or re-using a generic `INFERENCE_TIMEOUT_SEC`) and wiring it through `AnthropicInferenceConfig`.
How can I resolve this? If you propose a fix, please make it concise.| function resolveAnthropicModel(model: string, openAIDefault: string): string { | ||
| if (model === openAIDefault) { | ||
| logger.info( | ||
| `[inference] No Claude model set for the Anthropic provider; defaulting to ${ANTHROPIC_DEFAULT_MODEL}. Set INFERENCE_TEXT_MODEL/INFERENCE_IMAGE_MODEL to override.`, | ||
| ); | ||
| return ANTHROPIC_DEFAULT_MODEL; | ||
| } | ||
| return model; | ||
| } |
There was a problem hiding this comment.
resolveAnthropicModel is called for both textModel and imageModel, but the log message always says "No Claude model set … Set INFERENCE_TEXT_MODEL/INFERENCE_IMAGE_MODEL to override." — it never identifies which model slot triggered the substitution. When both are defaulted, users see the same message twice with no way to tell them apart. Passing a slot identifier (e.g. "text" / "image") would make the message actionable.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/shared/inference.ts
Line: 179-187
Comment:
**Ambiguous log message: text vs. image model substitution indistinguishable**
`resolveAnthropicModel` is called for both `textModel` and `imageModel`, but the log message always says "No Claude model set … Set INFERENCE_TEXT_MODEL/INFERENCE_IMAGE_MODEL to override." — it never identifies which model slot triggered the substitution. When both are defaulted, users see the same message twice with no way to tell them apart. Passing a `slot` identifier (e.g. `"text"` / `"image"`) would make the message actionable.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
| return new AnthropicInferenceClient({ | ||
| apiKey: "test-key", | ||
| textModel: "gpt-4.1-mini", | ||
| imageModel: "gpt-4o-mini", | ||
| maxOutputTokens: 100, | ||
| outputSchema: "structured", | ||
| ...overrides, | ||
| }); | ||
| } |
There was a problem hiding this comment.
makeClient() defaults textModel: "gpt-4.1-mini" and imageModel: "gpt-4o-mini". The tests exercise the substitution path (good), but the helper's name and intent could mislead a future contributor into thinking it simulates a misconfigured OpenAI client rather than an Anthropic client with OpenAI-defaulted models. A brief comment explaining why the OpenAI defaults are intentional here would improve clarity.
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/shared/inference.test.ts
Line: 17-25
Comment:
**Test fixture uses OpenAI model names for an Anthropic client under test**
`makeClient()` defaults `textModel: "gpt-4.1-mini"` and `imageModel: "gpt-4o-mini"`. The tests exercise the substitution path (good), but the helper's name and intent could mislead a future contributor into thinking it simulates a misconfigured OpenAI client rather than an Anthropic client with OpenAI-defaulted models. A brief comment explaining why the OpenAI defaults are intentional here would improve clarity.
How can I resolve this? If you propose a fix, please make it concise.Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
1 participant
Summary
Adds a first-class Anthropic (Claude) inference provider, selected by
ANTHROPIC_API_KEY, using the official@anthropic-ai/sdkMessages API.Today Claude can only be used by pointing
OPENAI_BASE_URLat Anthropic's OpenAI-compatibility endpoint, which ignoresstrict/json_schemaenforcement — so Karakeep's defaultstructuredtagging output isn't actually enforced, leading to occasional malformed JSON and failed tagging jobs. This native provider uses Anthropic's Structured Outputs for guaranteed schema conformance.What it does
AnthropicInferenceClientinpackages/shared/inference.ts, selected inInferenceClientFactory.build()whenANTHROPIC_API_KEYis set (precedence: OpenAI → Anthropic → Ollama).structured/json/plainoutput modes onto Anthropic'soutput_config.formatjson_schema, reusing the samez.toJSONSchemathe Ollama client already uses.claude-haiku-4-5when no Claude model is configured (override viaINFERENCE_TEXT_MODEL/INFERENCE_IMAGE_MODEL).ANTHROPIC_API_KEY, optionalANTHROPIC_BASE_URL. Docs updated under03-configuration.The change is purely additive — no existing OpenAI/Ollama logic is modified.
Limitations
generateEmbeddingFromTextthrows a clear, documented error. Semantic search still requires a separate embedding provider (OpenAI/Ollama).Test plan
packages/shared/inference.test.ts(8) cover text, image, structured-output mapping, model-default substitution, and the embeddings error.pnpm format,pnpm lint,pnpm typecheck,pnpm buildall pass.@karakeep/sharedtest suite green.🤖 Generated with Claude Code