Skip to content

Decide what else moves into packages/llm once a second gateway consumer exists #325

Open

Description

@ckrough

Follow-up from #93 (shared gateway client package), promised in PR #321. Deliberately deferred, not forgotten: it needs a second consumer to exist before the right shape is knowable.

#93 moved build_gateway_client into packages/llm but left the settings side in each service. retriever.config.Settings still owns:

  • llm_gateway_base_url (the computed field resolving LLM_GATEWAY_URL, else CLOUDFLARE_ACCOUNT_ID + CLOUDFLARE_GATEWAY_ID, else ValueError)
  • gateway_token_for(scope) (scoped-token lookup with fallback to the shared llm_gateway_token)
  • the gateway fields themselves (llm_gateway_token, llm_gateway_auth_header, the per-scope tokens)

GatewaySettings in packages/llm is a structural typing.Protocol describing that surface, not an implementation of it. So the second service to call the gateway must reimplement all three, which is the copy-paste #93 set out to prevent, one layer up.

This was the right call for #93: with one consumer, any shared base class would be designed against a sample size of one, and a pydantic settings mixin would have coupled the package to pydantic-settings for no present benefit. The question is genuinely open until a second service has real requirements.

Do this when a second service (biowriter, per ADR 0028, or petdata if it starts making model calls) lands its first gateway call. Resolve then:

  • Whether the shared piece is a pydantic-settings mixin, a plain resolver function taking the env values, or documentation plus the Protocol as-is.
  • Whether the base-URL resolution (LLM_GATEWAY_URL / Cloudflare account+gateway derivation) moves into packages/llm, given Validate gateway base URL scheme and host in build_gateway_client #323 will add validation there anyway.
  • Whether gateway_token_for's scoped-token-with-fallback logic moves, and if so how per-service env prefixes (PETDATA_, etc.) are handled.
  • Whether retriever migrates onto the shared piece in the same PR or a follow-up.

Acceptance Criteria

  • A decision is recorded, in packages/llm/README.md or an ADR amendment, even if the decision is to keep the Protocol-only split.
  • If shared code is added, no service reimplements base-URL resolution or scoped-token fallback.
  • packages/llm still imports nothing from any service (ADR 0001).
  • Retriever's gateway behavior is unchanged, verified by its existing scoped-token tests.

Activity

  1. added
    module:platformCross-cutting platform / foundation
    taskTests, docs, refactoring, non-feature work
    on Aug 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    module:platformCross-cutting platform / foundationtaskTests, docs, refactoring, non-feature work

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions