Skip to content

feat(cron): durable run-report observability + logging cleanup - #51

Merged
GJanos merged 3 commits into
developfrom
feat/cron-run-report
Jun 18, 2026
Merged

GJanos merged 3 commits into
developfrom
feat/cron-run-report

Conversation

@GJanos

@GJanos GJanos commented Jun 18, 2026 •

Copy link
Copy Markdown
Owner

Implements Phase 2 — Observability: cost logging refactor from todo.md.

What

Each daily-digest run now produces one complete, versioned RunReport (pulse.run.v1) and persists it to a new public.pipeline_runs table — a durable, queryable cost/health record. Until now the structured run object was only appended to a JSONL file on the ephemeral GitHub Actions runner and thrown away.

Run report (cron/src/runReport.ts)

  • buildRunReport(...) — pure aggregation: fetch cost + per-region ranking + the global-ranking pass (whose usage was previously dropped entirely), region partition (requested/succeeded/failed), run status (ok/partial), headline counts, and notify outcome.
  • formatRunReportSummary(...) — one dense, human-readable summary line:
    Run [ok] — 9/9 regions, 312 headlines | fetch $0.0461 (45.2k tok) | ranking $0.0123 (8.1k tok) | total $0.0584 | notify 12/12 | 142.3s
    
  • persistRunReport(...) — inserts via the service-role client; swallows any error (logs a warning) so observability can never break the digest pipeline.
  • rankGlobalHeadlines now returns { headlines, usage } and accumulates usage across all passes (the only non-trivial signature change; callers + e2e runners updated).

Database

public.pipeline_runs — promoted scalar columns (run_at, status, total_cost_usd, total_tokens, regions_*) for cheap cost-over-time queries, plus the full structured report as jsonb. RLS enabled, no policy → service-role-only (matches the existing private devices/notify_state pattern; advisor confirms only the expected rls_enabled_no_policy INFO, no new warnings). No sensitive data at rest (no tokens / user IDs / secrets).

  • Migration applied to the live project (create_pipeline_runs).
  • Mirrored into supabase/schema.sql.
SELECT run_at, status, total_cost_usd, total_tokens
  FROM pipeline_runs ORDER BY run_at DESC LIMIT 30;

Logging cleanup across cron/

  • Stack traces preserved: new formatError(); both job entrypoints and the region-failure log used String(err), which dropped the stack.
  • TTY-aware color: colorize only on a real TTY, so GitHub Actions logs aren't littered with ANSI escape codes.
  • Run-lifecycle brackets: a clear start line in both jobs; the summary line closes the daily run.
  • Level correction: per-headline "Slug duplicate filtered" demoted info → debug.

Docs

  • Refreshed all four READMEs: run-report observability, pipeline_runs/notify_state tables + full RLS matrix + SECURITY DEFINER RPC reference (supabase), ranking + GHA cron jobs + catch-up window (cron/root), expo-linking deep-link handling + publishable-key env (app), Vercel→GitHub Actions corrections.
  • Removed the docs/superpowers/specs design-spec artifacts (planning byproducts).

Drive-by

  • Fixed a stale api dir reference (removed in an earlier PR) in both the cron lint script and the root lint-staged config — the latter was failing every cron pre-commit. Now lints src jobs e2e.
  • Dropped two completed items from todo.md.

Verification

  • npx tsc --noEmit (cron) — clean · eslint — 0 errors · npx jest (cron) — 184 tests pass · prettier --check — clean.

🤖 Generated with Claude Code

GJanos and others added 2 commits June 18, 2026 09:51
Each daily-digest run now emits one complete, versioned RunReport
('pulse.run.v1') and persists it to public.pipeline_runs — a durable,
queryable cost/health record. Previously the structured run object was only
appended to an ephemeral JSONL file on the Actions runner and discarded.

Run report:
- runReport.ts: buildRunReport (pure aggregation), formatRunReportSummary
  (one readable summary line), persistRunReport (service-role insert; swallows
  errors so observability never breaks the pipeline).
- Captures ALL cost: fetch + per-region ranking + the global-ranking pass,
  whose usage was previously dropped — rankGlobalHeadlines now returns
  { headlines, usage } and accumulates usage across passes.
- Folds in notify outcome (devicesTargeted/sent/failed) and run status.
- public.pipeline_runs: promoted scalar cost columns + full report jsonb,
  RLS-enabled with no policy (service-role only). Mirrored in supabase/schema.sql.

Logging cleanup across cron/:
- formatError() preserves Error stack traces; both job entrypoints and the
  region-failure log previously used String(err), dropping the stack.
- Colorize only on a TTY — keeps GitHub Actions logs free of ANSI noise.
- Run-lifecycle start brackets in both jobs; per-headline slug-duplicate log
  demoted info -> debug.

Also: fix stale 'api' dir reference in the cron lint script and the root
lint-staged config (the dir was removed earlier); drop completed todo items.

Tests: runReport (build/format/persist) + updated rankHeadlines for the new
return shape. cron suite green (184 tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…artifacts

READMEs:
- root: Vercel→GitHub Actions, Claude ranking, pipeline_runs observability row,
  device RLS-via-RPC lockdown, shared/ subsystem link.
- cron: ranking + run-report observability section, ANTHROPIC_API_KEY, the two
  GHA cron jobs and the notify catch-up window, updated pipeline diagram.
- supabase: pipeline_runs and notify_state tables, full RLS matrix (incl. the
  service-role-only cron tables), SECURITY DEFINER RPC table + anon auto-grant
  gotcha, UNIQUE/ON DELETE details.
- app: expo-linking deep-link handling (reset-password + signup-confirm),
  publishable-key env var, navigation model, trimmed stale WSL setup.

Also removes the docs/superpowers/specs design-spec artifacts (planning
byproducts) from the tree.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@GJanos
GJanos force-pushed the feat/cron-run-report branch from 0f376b2 to c57dea9 Compare June 18, 2026 07:57
app/README listed EXPO_PUBLIC_SUPABASE_PUBLISHABLE_KEY, but the client reads
EXPO_PUBLIC_SUPABASE_KEY (app/src/supabase/client.ts). Fix the name.

Also remove the docs/superpowers/plans planning artifacts.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@GJanos
GJanos merged commit 1707aac into develop Jun 18, 2026
5 checks passed
@GJanos
GJanos deleted the feat/cron-run-report branch June 18, 2026 08:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant