Skip to content

feat(data): add PostgresSource for live config reads#900

Draft
zulkhair wants to merge 5 commits into
mainfrom
daim/adr057-postgres-source
Draft

feat(data): add PostgresSource for live config reads#900
zulkhair wants to merge 5 commits into
mainfrom
daim/adr057-postgres-source

Conversation

@zulkhair

@zulkhair zulkhair commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Summary

The data plugin loads each config kind from build-time embedded JSON (EmbedSource). This adds PostgresSource, an alternative Source that reads config from a Postgres database, selected at runtime by the CONFIG_DB_DSN env var — so a deployment can change config without rebuilding and redeploying.

When CONFIG_DB_DSN is set the plugin reads config only from Postgres — fail-loud: a missing table is an error, with no silent fallback to embedded JSON. When it's unset, the embedded path is unchanged.

Table contract

Each config kind is read from its own table, where one row holds one record as JSON in a doc jsonb column:

  • Array kinds (a collection of records): SELECT coalesce(json_agg(doc ORDER BY id), '[]'::json)::text FROM <table> — rows aggregated into a JSON array, ordered by id for a stable content hash.
  • Single-object kinds (the whole config is one object): SELECT doc::text FROM <table> LIMIT 1 — one JSON object. A kind opts into this by implementing the Singleton marker interface (SingleObject()).

The table name for each kind — and whether it's a singleton — is registered when the kind registers: Register[T] calls RegisterKind(JSONFile(), Name()) and RegisterSingleton(...) for Singleton kinds. So a kind reads from a table named after its Name(), and shard main.go stays unchanged.

Design notes

  • Single-version, like EmbedSource: it ignores the requested content hash and returns current rows, so a snapshot restore resumes on current config (Reconcile's warn path) instead of erroring (the panic path).
  • PickSource centralizes the choice (CONFIG_DB_DSN set → PostgresSource, else EmbedSource); a set-but-unusable DSN panics — fail loud rather than silently serve stale embedded config.
  • A small configReader interface seam makes Fetch unit-testable without a live database.
  • Backed by a pgxpool; the pool connects lazily (a malformed DSN fails at construction, connectivity fails on first Fetch).

Testing

pkg/plugin/data/system/postgres_internal_test.go (fake configReader, no database): table-name resolution + registrar override, single-object vs array read shapes, rows + sha256 hash return, single-version (requested hash ignored), and error propagation.

Compatibility

Additive. The default path (no CONFIG_DB_DSN) is the unchanged EmbedSource. New dependencies: jackc/pgx/v5 + pgxpool.

zulkhair added 5 commits June 11, 2026 21:07
…ray reads

json_agg returns json, so coalescing it with a '[]'::jsonb fallback errored
(SQLSTATE 42846) on every non-empty config table. Use a '[]'::json fallback so
array kinds read back; output text is unchanged.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant