generate cron refactoring

.docs/_cron.md

@@ -3,32 +3,46 @@
 You have to allow two extensions: `pg_cron`, `pg_net` for a database to make
 this work. See [Supabase Documentation](https://supabase.com/docs/guides/cron).
 
-You have to run the following commands in SQL Editor of your Supabase project.
+## How cron jobs are deployed
 
-## Generate cron tab
+Cron jobs are defined in code and deployed automatically as SQL migrations via
+`supabase db push` in CI. No manual SQL editor steps required.
 
-1. Fill out root `.env` file with variables
-2. run tests: `deno run tests`
-3. cron install queries will be generated into the file `cron_jobs.sql` in the
-   root
-4. visit supabase dashboard and run these queries in SQL editor
+The anon key is **never stored in the repository**. It is fetched from the
+Supabase Management API at deploy time and stored in Supabase Vault
+(`vault.secrets`), where pg_cron reads it at runtime.
 
-## GUI
+See [decision record 003](decision-records/003-cron-vault-secrets.md) for full details.
 
-There is actually a UI where you can manage your jobs very easily.
+## Adding or changing a cron job
 
-```
-https://supabase.com/dashboard/project/***/integrations/cron/jobs
-```
+1. Edit the `CronDefinition` in the relevant Edge Function:
+   - `supabase/functions/lastfm-user-recent-tracks/sync-tracks.ts` → `recentTracksCron`
+   - `supabase/functions/lastfm-library-artists/sync-artists.ts` → `libraryArtistsCron`
+2. Add or remove users in `supabase/functions/_shared/users.ts` → `LASTFM_USERS`
+3. Generate and stage the migration:
+   ```bash
+   make cron
+   ```
+4. Commit and push — CI applies it via `supabase db push`
 
-_Replace your project ID in the URL_.
+## Local development
 
-## Remove existing job
-
-Replace ID by number from your database of already existing cron.
+Seed the vault secret once after starting local Supabase:
 
 ```bash
-select cron.unschedule(3);
+# Get the local anon key from:
+deno task start  # then check output of: supabase status
+
+psql $DATABASE_URL -c "SELECT vault.create_secret('<local-anon-key>', 'supabase_anon_key', 'Supabase anon key for cron HTTP requests');"
+```
+
+## GUI
+
+Jobs can also be inspected and managed via the Supabase dashboard:
+
+```
+https://supabase.com/dashboard/project/***/integrations/cron/jobs
 ```
 
-Or you can do this via GUI as well.
+_Replace your project ID in the URL._

.docs/decision-records/000-TEMPLATE.md

@@ -0,0 +1,23 @@
+# Title
+
+<!-- Short, descriptive title of the decision. Avoid "architecture" or "ADR" in the name. -->
+
+## Status
+
+<!-- proposed | accepted | rejected | deprecated | superseded by [NNN](NNN-title.md) -->
+
+Proposed
+
+## Context
+
+<!-- What problem or situation is motivating this decision?
+     What constraints, requirements or observations led here? -->
+
+## Decision
+
+<!-- What is being decided? What change is proposed or already made? -->
+
+## Consequences
+
+<!-- What becomes easier or harder after this decision?
+     List both positive and negative consequences. -->

.docs/decision-records/003-cron-vault-secrets.md

@@ -0,0 +1,91 @@
+# 003 — Cron Jobs Use Supabase Vault for Secrets (No Hardcoded Keys in Public Repo)
+
+**Date:** 2026-03-27
+**Status:** Active
+
+---
+
+## Problem
+
+The `cron_jobs.sql` file contained the Supabase `anon` key hardcoded as a bearer token in the HTTP headers of every `net.http_post` call. This file was committed to a public GitHub repository, exposing the key publicly.
+
+Additionally, cron jobs were not deployed automatically — `cron_jobs.sql` was a manual artifact that had to be applied by hand, with no CI integration.
+
+---
+
+## Decision
+
+Store the Supabase `anon` key in **Supabase Vault** (`vault.secrets`) and reference it at cron execution time via `vault.decrypted_secrets`. Cron job definitions live as a standard migration under `supabase/migrations/` and are applied automatically via `supabase db push` in CI.
+
+The vault secret is upserted by the CI pipeline itself (via `psql` on the self-hosted runner) — before `supabase db push` runs. The anon key is fetched at deploy time from the Supabase Management API using `SUPABASE_ACCESS_TOKEN` (already required for deployment), so no additional GitHub secret is needed. The migration file contains no secret values and is safe to commit.
+
+---
+
+## Architecture
+
+```
+Supabase Management API
+  └── projects api-keys (fetched via SUPABASE_ACCESS_TOKEN at deploy time)
+        │
+        ▼  (psql upsert step, before db push)
+Supabase Vault
+  └── vault.secrets { name: 'supabase_anon_key', secret: '<anon key>' }
+        │
+        ▼  (read at cron execution time by pg_cron)
+pg_cron job SQL
+  └── SELECT decrypted_secret FROM vault.decrypted_secrets WHERE name = 'supabase_anon_key'
+```
+
+The migration file stores the cron SQL as a string. pg_cron evaluates it at runtime, so the vault lookup happens each time the cron fires — not at migration time.
+
+---
+
+## Why anon key and not service_role key
+
+Cron jobs call Edge Functions via their public HTTPS endpoint. Edge Functions authenticate callers using the anon key (JWT), not the service_role key. The service_role key bypasses RLS at the DB level and is never needed for HTTP-level invocation. Only the anon key is stored in vault for cron use.
+
+---
+
+## Changing Crons Over Time
+
+Cron definitions live in `CronDefinition` objects in each Edge Function's sync file. Users are defined in `supabase/functions/_shared/users.ts`.
+
+After any change, generate and stage a new migration:
+
+```bash
+make cron
+```
+
+This runs `scripts/generate-crons.ts`, which writes a timestamped migration to `supabase/migrations/` and prints the path. The Makefile stages it with `git add`. Every generated migration unschedules all existing jobs first, then reschedules them — so no stale jobs can accumulate.
+
+---
+
+## Local Development
+
+When working locally with Supabase, seed the vault secret once:
+
+```bash
+psql $DATABASE_URL -c "SELECT vault.create_secret('your-local-anon-key', 'supabase_anon_key', 'Supabase anon key for cron HTTP requests');"
+```
+
+The local anon key is found in the output of `supabase status` after `deno task start`.
+
+---
+
+## Security Properties
+
+| Property                        | Before                 | After                             |
+|---------------------------------|------------------------|-----------------------------------|
+| Anon key in public repo         | Yes (hardcoded in SQL) | No                                |
+| Anon key in CI logs             | N/A                    | No (GitHub masked secret)         |
+| Anon key at rest                | Plaintext in git       | Encrypted in vault (pgsodium)     |
+| Cron deployment                 | Manual                 | Automatic via `supabase db push`  |
+| Self-hosted runner IP whitelist | N/A                    | DB accessible only from static IP |
+
+---
+
+## Alternatives Considered
+
+**Template + envsubst in CI** — Replace `${SUPABASE_ANON_KEY}` at CI time and generate the migration file. Rejected: adds fragility (file generation in CI), and the secret is still handled in plaintext in the shell. Vault keeps the secret encrypted at rest in the DB itself.
+
+**`current_setting()` via `ALTER DATABASE SET`** — Store key as a DB-level setting. Rejected: not encrypted at rest; visible in `pg_settings`.

.docs/decision-records/README.md

@@ -0,0 +1,5 @@
+# Decision Records
+
+- [001](001-artist-upsert-anon-role.md) — Artist upsert performance regression caused by missing service role key (anon role fallback)
+- [002](002-reduce-egress.md) — Supabase egress reduction after hitting free-tier bandwidth limit
+- [003](003-cron-vault-secrets.md) — Cron jobs use Supabase Vault for secrets; no hardcoded keys in public repo

.github/workflows/deploy-supabase.yml

@@ -38,6 +38,29 @@ jobs:
           npx @sentry/cli releases set-commits ${{ github.sha }} --auto
           npx @sentry/cli releases deploys ${{ github.sha }} new -e production
 
+      - name: "Supabase - Upsert vault secrets"
+        env:
+          PROJECT_ID: ${{ secrets.SUPABASE_PROJECT_ID }}
+          SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }}
+          SUPABASE_DB_PASSWORD: ${{ secrets.SUPABASE_DB_PASSWORD }}
+        shell: bash
+        run: |
+          ANON_KEY=$(supabase projects api-keys --project-ref "$PROJECT_ID" --output json | jq -r '.[] | select(.name == "anon") | .api_key')
+          DATABASE_URL="postgresql://postgres:${SUPABASE_DB_PASSWORD}@db.${PROJECT_ID}.supabase.co:5432/postgres"
+          psql "$DATABASE_URL" <<EOF
+          DO \$\$
+          DECLARE
+            v_id uuid;
+          BEGIN
+            SELECT id INTO v_id FROM vault.secrets WHERE name = 'supabase_anon_key' LIMIT 1;
+            IF v_id IS NOT NULL THEN
+              PERFORM vault.update_secret(v_id, '${ANON_KEY}');
+            ELSE
+              PERFORM vault.create_secret('${ANON_KEY}', 'supabase_anon_key', 'Supabase anon key for pg_cron HTTP requests');
+            END IF;
+          END \$\$;
+          EOF
+
       - name: "Supabase - Remote Connect"
         env:
           PROJECT_ID: ${{ secrets.SUPABASE_PROJECT_ID }}

.mcp.json

@@ -1,6 +1,7 @@
 {
   "mcpServers": {
     "supabase": {
+      "type": "http",
       "url": "https://mcp.supabase.com/mcp"
     }
   }

Makefile

@@ -1,6 +1,11 @@
+# Paths
 WEB := src/web
-GREEN := \033[0;32m
-END := \033[0m
+
+# Colors
+RESET  := \033[0m
+CYAN   := \033[36m
+GREEN  := \033[1;32m
+GRAY   := \033[90m
 
 -include .env
 export
@@ -40,6 +45,7 @@ lint:
 # DOC
 #
 .PHONY: decision
+DOC_DECISIONS_FOLDER := .docs/decision-records
 
 decision: ## Create a new decision record (usage: make decision, or make decision TITLE="my title")
 	@if [ -n "$(TITLE)" ]; then \
@@ -53,14 +59,12 @@ decision: ## Create a new decision record (usage: make decision, or make decisio
 		sed 's/[^a-z0-9 -]//g' | \
 		tr ' ' '-' | \
 		sed 's/-\{2,\}/-/g; s/^-//; s/-$$//'); \
-	last=$$(ls .docs/decision-records/[0-9][0-9][0-9]-*.md 2>/dev/null | sort | tail -1); \
-	if [ -z "$$last" ]; then next="001"; \
-	else next=$$(basename "$$last" | sed 's/-.*//' | awk '{printf "%03d", $$1 + 1}'); fi; \
-	file=".docs/decision-records/$${next}-$${slug}.md"; \
-	cp .docs/decision-records/000-TEMPLATE.md "$$file"; \
+	next=$$(date +%Y%m%d-%H%M); \
+	file="$(DOC_DECISIONS_FOLDER)/$${next}-$${slug}.md"; \
+	cp $(DOC_DECISIONS_FOLDER)/000-TEMPLATE.md "$$file"; \
 	basename=$$(basename "$$file"); \
-	echo "- [$${next}]($$basename) — $$title" >> .docs/decision-records/README.md; \
-	git add "$$file" .docs/decision-records/README.md; \
+	echo "- [$${next}]($$basename) — $$title" >> $(DOC_DECISIONS_FOLDER)/README.md; \
+	git add "$$file" $(DOC_DECISIONS_FOLDER)/README.md; \
 	echo "$(GREEN)Created: $$file$(END)"; \
 	if [ -z "$(TITLE)" ]; then \
 		case "$$(uname)" in \
@@ -117,11 +121,16 @@ tests:
 	deno task tests
 
 # Database
-.PHONY: migration migrate update-types-local update-types-linked
+.PHONY: migration cron migrate update-types-local update-types-linked
 
 migration:
 	deno task migration $(name)
 
+cron: ## Generate cron migration from source (requires PROJECT_ID in .env)
+	@file=$$(deno run --allow-write=supabase/migrations --allow-env=PROJECT_ID scripts/generate-crons.ts); \
+	git add "$$file"; \
+	echo "$(GREEN)Created:$(RESET) file://$(CURDIR)/$$file"
+
 migrate:
 	deno task migrate
 

scripts/generate-crons.ts

@@ -0,0 +1,30 @@
+import { buildCron, buildUnschedule } from "../supabase/functions/_shared/cron.ts";
+import { recentTracksCron } from "../supabase/functions/lastfm-user-recent-tracks/sync-tracks.ts";
+import { libraryArtistsCron } from "../supabase/functions/lastfm-library-artists/sync-artists.ts";
+import { LASTFM_USERS } from "../supabase/functions/_shared/users.ts";
+
+const PROJECT_ID = Deno.env.get("PROJECT_ID");
+if (!PROJECT_ID) throw new Error("Missing environment variable: PROJECT_ID");
+
+const unschedule = LASTFM_USERS.flatMap((user) => [
+  buildUnschedule(recentTracksCron, user),
+  buildUnschedule(libraryArtistsCron, user),
+]);
+
+const schedule = LASTFM_USERS.flatMap((user) => [
+  buildCron(recentTracksCron, PROJECT_ID, user),
+  buildCron(libraryArtistsCron, PROJECT_ID, user),
+]);
+
+const output = [
+  "-- unschedule all existing cron jobs before reapplying",
+  unschedule.join(""),
+  "-- schedule cron jobs",
+  schedule.join("\n"),
+].join("\n");
+
+const timestamp = new Date().toISOString().replace(/\D/g, "").slice(0, 14);
+const migrationFile = `supabase/migrations/${timestamp}_cron-jobs.sql`;
+
+await Deno.writeTextFile(migrationFile, output);
+console.log(migrationFile);