feat(umami-postgres): keploy compat lane sample (smoke-test only)

.github/workflows/scripts/run-and-measure.sh

@@ -0,0 +1,112 @@
+#!/usr/bin/env bash
+#
+# run-and-measure.sh — bring umami up via the sample's compose,
+# run flow.sh bootstrap + record-traffic with the per-call audit
+# log enabled, run flow.sh coverage, and emit `coverage=PCT`
+# onto $GITHUB_OUTPUT for the downstream coverage-gate job.
+#
+# Called from .github/workflows/umami-postgres.yml's
+# build-coverage and release-coverage jobs (one per ref under
+# comparison). Both jobs source the same script so the
+# measurement is identical across refs — any drift in the
+# numerator definition would otherwise produce a misleading
+# delta.
+#
+# Inputs (all from the workflow env):
+#   UMAMI_FIRED_ROUTES_FILE   — per-call audit log path; passed
+#                               through to flow.sh so its
+#                               record-traffic loop logs each
+#                               (METHOD, URL) pair, and so its
+#                               coverage subcommand uses that
+#                               file as the standalone
+#                               numerator.
+#   UMAMI_PHASE               — label spliced into the project
+#                               name and the on-disk token path
+#                               (`/tmp/umami-token-${UMAMI_PHASE}`)
+#                               so build vs. release runs don't
+#                               collide on volume names or token
+#                               files. Compose project naming
+#                               inside the GH runner is per-job
+#                               anyway, but UMAMI_PHASE is
+#                               useful for diffing logs.
+#   GITHUB_OUTPUT             — standard GH Actions sink for
+#                               step outputs.
+set -Eeuo pipefail
+
+export UMAMI_APP_CONTAINER="${UMAMI_APP_CONTAINER:-umami_app}"
+export UMAMI_DB_CONTAINER="${UMAMI_DB_CONTAINER:-umami_db}"
+export UMAMI_APP_PORT="${UMAMI_APP_PORT:-3001}"
+export UMAMI_APP_SECRET="${UMAMI_APP_SECRET:-keploy-fixed-app-secret-for-deterministic-recordings}"
+: "${UMAMI_FIRED_ROUTES_FILE:?UMAMI_FIRED_ROUTES_FILE must be set by the workflow}"
+
+# Reset audit log for this run; otherwise a prior run's entries
+# would inflate the numerator on a re-trigger.
+: >"$UMAMI_FIRED_ROUTES_FILE"
+
+# Stage 1: cold boot — umami's entrypoint runs Prisma migrations
+# + seeds the admin user into the named volume. UMAMI_SKIP_INIT=0
+# means "do the init work this time."
+UMAMI_SKIP_INIT=0 docker compose up -d
+
+# Wait for the backend to actually serve. /api/heartbeat returns
+# 200 only when Next.js has bound and Prisma is connected — a
+# stronger gate than wait-for-port, since umami is up on :3000
+# inside the container before the Next.js server has finished
+# warming the route table.
+for i in $(seq 1 120); do
+    code=$(curl -sS -o /dev/null -w '%{http_code}' \
+        "http://127.0.0.1:${UMAMI_APP_PORT}/api/heartbeat" 2>/dev/null || echo "")
+    if [ "$code" = "200" ]; then break; fi
+    sleep 2
+done
+
+bash flow.sh bootstrap 240
+docker compose down --remove-orphans
+
+# Stage 2: re-launch in skip-init mode against the populated
+# volume — same shape the keploy lanes use, so the recorded
+# request stream matches what record/replay sees.
+UMAMI_SKIP_INIT=1 docker compose up -d
+
+# Wait again — same readiness gate. Stage 2 is faster than stage
+# 1 (no migrations) but Next.js still needs ~10-30s to warm.
+for i in $(seq 1 120); do
+    code=$(curl -sS -o /dev/null -w '%{http_code}' \
+        "http://127.0.0.1:${UMAMI_APP_PORT}/api/heartbeat" 2>/dev/null || echo "")
+    if [ "$code" = "200" ]; then break; fi
+    sleep 2
+done
+
+# Re-bootstrap: the auth token is request-scoped (JWT in the
+# Authorization header), and stage 1's compose-down dropped the
+# in-memory token. flow.sh::umami_bootstrap re-issues a fresh
+# one against the same admin credentials and rewrites
+# /tmp/umami-token-${UMAMI_PHASE}, which umami_record_traffic
+# reads.
+bash flow.sh bootstrap 240
+
+# Drive traffic. flow.sh::umami_record_traffic re-reads the
+# token from /tmp/umami-token-${UMAMI_PHASE} and tolerates
+# non-2xx responses internally, so a single endpoint regression
+# in umami itself doesn't abort the whole record run.
+bash flow.sh record-traffic
+
+# Coverage report — uses UMAMI_FIRED_ROUTES_FILE as numerator
+# since no keploy/test-set-* tree exists in the standalone case.
+# umami_list_routes walks src/app/api/**/route.ts inside the
+# running container, so the app must still be up here.
+COVERAGE_REPORT_FILE="$PWD/coverage_report.txt" bash flow.sh coverage
+
+# Pull the percentage out of the report's `Covered N/M (XX.X%)`
+# line. Anchored on the parenthesised form so a future change to
+# the report's prose doesn't break the parse.
+pct=$(grep -oE '\([0-9]+\.[0-9]+%\)' coverage_report.txt | head -1 | tr -d '()%')
+if [ -z "$pct" ]; then
+    echo "::error::Could not parse coverage percentage from coverage_report.txt"
+    cat coverage_report.txt || true
+    exit 1
+fi
+echo "coverage=${pct}" >>"$GITHUB_OUTPUT"
+echo "coverage: ${pct}% (audit log: $UMAMI_FIRED_ROUTES_FILE)"
+
+docker compose down -v --remove-orphans

.github/workflows/umami-postgres.yml

@@ -0,0 +1,199 @@
+# umami-postgres sample CI — keploy-independent end-to-end smoke +
+# coverage gate.
+#
+# Triggers ONLY on changes under umami-postgres/ (or this workflow
+# file). Other samples in this repo have their own orthogonal CI;
+# gating the whole repo on every umami change would slow them
+# all down for no benefit.
+#
+# What it gates:
+#   * `release-coverage` — checks out the PR's base branch (main)
+#     and runs the sample end-to-end: docker compose up, bootstrap
+#     admin token, drive flow.sh record-traffic with the per-call
+#     audit log enabled, capture the route-coverage percentage from
+#     `flow.sh coverage`. This is the baseline.
+#   * `build-coverage` — same end-to-end against the PR's HEAD ref.
+#   * `coverage-gate` — fails the PR if `build`'s coverage drops
+#     more than COVERAGE_THRESHOLD percentage points below
+#     `release`. Default threshold is 1.0pp; override via repo
+#     variable `UMAMI_COVERAGE_THRESHOLD` for a tighter or
+#     looser bar.
+#
+# On push to main, only `build-coverage` runs (no baseline to
+# compare against — main IS the baseline).
+#
+# Standards-aligned choices:
+#   * `paths:` filter on both push and pull_request triggers — the
+#     canonical GH Actions way to scope a workflow to one
+#     subdirectory.
+#   * Job outputs (steps.<id>.outputs.coverage → needs.<job>.outputs)
+#     to thread the captured percentage between jobs.
+#   * `concurrency:` cancel-in-progress on the same ref so a stale
+#     run doesn't waste runner minutes.
+#   * actions/upload-artifact for the human-readable
+#     coverage_report.txt — reviewers can inspect missing routes
+#     directly from the PR's "checks" tab.
+#   * marocchino/sticky-pull-request-comment for the PR-side diff
+#     comment. Pinned-by-header so successive runs update the same
+#     comment instead of fanning out.
+#   * The compare step is plain bash + python3 (no external
+#     coverage service). For full coverage XMLs you'd want
+#     diff-cover or codecov, but the sample's coverage is
+#     API-route-based (single percentage), so the gate is a 3-line
+#     subtraction.
+#
+# Sample is genuinely keploy-independent here: the workflow uses
+# flow.sh's $UMAMI_FIRED_ROUTES_FILE per-call audit log as its
+# numerator source, not a keploy recording. The lane scripts in
+# keploy/integrations and keploy/enterprise consume the same
+# flow.sh, but use the keploy/test-set-*/tests/*.yaml tree as
+# their numerator (authoritative — only calls keploy actually
+# CAPTURED count). Both modes are wired into
+# `flow.sh::umami_list_recorded_routes`.
+name: umami-postgres sample
+
+on:
+  pull_request:
+    paths:
+      - 'umami-postgres/**'
+      - '.github/workflows/umami-postgres.yml'
+  push:
+    branches: [main]
+    paths:
+      - 'umami-postgres/**'
+      - '.github/workflows/umami-postgres.yml'
+  workflow_dispatch: {}
+
+concurrency:
+  group: umami-postgres-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  COVERAGE_THRESHOLD: ${{ vars.UMAMI_COVERAGE_THRESHOLD || '1.0' }}
+
+jobs:
+  build-coverage:
+    name: build (current ref) coverage
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    outputs:
+      coverage: ${{ steps.measure.outputs.coverage }}
+    steps:
+      - uses: actions/checkout@v4
+      - id: measure
+        name: Run sample end-to-end + measure coverage
+        working-directory: umami-postgres
+        env:
+          UMAMI_FIRED_ROUTES_FILE: ${{ runner.temp }}/fired-routes-build.log
+          UMAMI_PHASE: ci-build
+        run: ../.github/workflows/scripts/run-and-measure.sh
+
+      - name: Upload coverage report
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-build
+          path: umami-postgres/coverage_report.txt
+          if-no-files-found: warn
+
+  release-coverage:
+    if: github.event_name == 'pull_request'
+    name: release (base ref) coverage
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
+    outputs:
+      coverage: ${{ steps.measure.outputs.coverage || steps.empty-baseline.outputs.coverage }}
+      sample-existed: ${{ steps.detect.outputs.sample-existed }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event.pull_request.base.ref }}
+
+      # First-PR bootstrap escape hatch: the very PR that
+      # introduces the umami-postgres/ sample has no baseline
+      # (umami-postgres/ doesn't exist on the base ref). Detect
+      # that and short-circuit to coverage=0; the gate then
+      # treats build's coverage as the new baseline and trivially
+      # passes for any percentage > 0. After the introducing PR
+      # merges, every subsequent PR has a real baseline to diff
+      # against.
+      - id: detect
+        name: Detect baseline presence
+        run: |
+          if [ -d umami-postgres ] && [ -x umami-postgres/flow.sh ]; then
+            echo "sample-existed=true" >>"$GITHUB_OUTPUT"
+            echo "Sample exists on base ref — running full measurement."
+          else
+            echo "sample-existed=false" >>"$GITHUB_OUTPUT"
+            echo "No umami-postgres/ on base ref — first-PR bootstrap; baseline coverage treated as 0%."
+          fi
+
+      - id: measure
+        name: Run sample end-to-end + measure coverage
+        if: steps.detect.outputs.sample-existed == 'true'
+        working-directory: umami-postgres
+        env:
+          UMAMI_FIRED_ROUTES_FILE: ${{ runner.temp }}/fired-routes-release.log
+          UMAMI_PHASE: ci-release
+        run: ../.github/workflows/scripts/run-and-measure.sh
+
+      - id: empty-baseline
+        name: Emit zero baseline (first-PR bootstrap)
+        if: steps.detect.outputs.sample-existed != 'true'
+        run: echo "coverage=0.0" >>"$GITHUB_OUTPUT"
+
+      - name: Upload coverage report
+        if: always() && steps.detect.outputs.sample-existed == 'true'
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-release
+          path: umami-postgres/coverage_report.txt
+          if-no-files-found: warn
+
+  coverage-gate:
+    if: github.event_name == 'pull_request'
+    name: coverage gate
+    needs: [build-coverage, release-coverage]
+    runs-on: ubuntu-latest
+    steps:
+      - name: Compare build vs release
+        env:
+          BUILD: ${{ needs.build-coverage.outputs.coverage }}
+          RELEASE: ${{ needs.release-coverage.outputs.coverage }}
+          THRESHOLD: ${{ env.COVERAGE_THRESHOLD }}
+          BASE_REF: ${{ github.event.pull_request.base.ref }}
+        run: |
+          set -Eeuo pipefail
+          if [ -z "${BUILD:-}" ] || [ -z "${RELEASE:-}" ]; then
+            echo "::error::missing coverage outputs — build='${BUILD:-}' release='${RELEASE:-}'"
+            exit 1
+          fi
+          drop=$(python3 -c "print(round(${RELEASE} - ${BUILD}, 2))")
+          echo "Release (${BASE_REF}): ${RELEASE}%"
+          echo "Build   (this PR):     ${BUILD}%"
+          echo "Drop:                  ${drop}pp (threshold ${THRESHOLD}pp)"
+          if python3 -c "import sys; sys.exit(0 if (${RELEASE} - ${BUILD}) > ${THRESHOLD} else 1)"; then
+            echo "::error::umami-postgres coverage dropped from ${RELEASE}% → ${BUILD}% (-${drop}pp), exceeding the ${THRESHOLD}pp threshold."
+            echo "Suggested actions:"
+            echo "  * Add curl(s) to flow.sh::umami_record_traffic that exercise the routes you changed/touched."
+            echo "  * If the route(s) was intentionally retired, drop it from umami-postgres/flow.sh::umami_list_routes' route-table walk too so it's removed from the denominator."
+            exit 1
+          fi
+          echo "OK — coverage delta within ${THRESHOLD}pp threshold."
+
+      - name: Sticky PR comment
+        if: ${{ !cancelled() }}
+        uses: marocchino/sticky-pull-request-comment@v2
+        with:
+          header: umami-postgres-coverage
+          message: |
+            ### umami-postgres sample coverage
+
+            | ref | coverage |
+            |---|---|
+            | base (`${{ github.event.pull_request.base.ref }}`) | **${{ needs.release-coverage.outputs.coverage }}%** |
+            | this PR | **${{ needs.build-coverage.outputs.coverage }}%** |
+
+            Threshold: PR may not drop coverage by more than **${{ env.COVERAGE_THRESHOLD }}pp**. Override per-repo via the `UMAMI_COVERAGE_THRESHOLD` actions variable.
+
+            Coverage measures the umami v2 API surface (`/api/auth/*` + `/api/me` + `/api/users` + `/api/teams` + `/api/websites/*` + `/api/reports/*` + `/api/share/*` + heartbeat) that `flow.sh::umami_record_traffic` actually exercises against the running backend. Reports are attached as artifacts on each job ("coverage-build" / "coverage-release").

umami-postgres/Dockerfile

@@ -0,0 +1,9 @@
+# Thin wrapper around umami's official image at the version this
+# sample tracks. Pin lives here (not in CI lane scripts) so a
+# future umami release that changes the bug-triggering shape is a
+# one-line retag, not a hunt across keploy/integrations and
+# keploy/enterprise.
+#
+# Upstream: https://github.com/umami-software/umami
+# Image:    docker.io/umamisoftware/umami:postgresql-v2.18.1
+FROM ghcr.io/umami-software/umami:postgresql-v2.18.1

umami-postgres/README.md

@@ -0,0 +1,42 @@
+# umami-postgres — keploy compat lane sample
+
+Reproducer for the umami / postgres-v3 compat lane. Mirrors the architectural pattern of the [doccano-django sample in `samples-python`](https://github.com/keploy/samples-python/tree/main/doccano-django): the sample owns orchestration (compose / bootstrap / traffic / noise filter / coverage), the keploy CI lanes consume it as a thin wrapper.
+
+The sample drives the full umami v2 API surface keploy needs to gate on a record/replay round-trip — auth + me + admin lists, users CRUD, websites CRUD, all eight report types, share tokens + public share access, batch + identify event ingest, sessions deep-dive, replays, boards lifecycle, pixel tracker, metric/pageview parser-branch variants, and logout.
+
+## Layout
+
+```
+umami-postgres/
+├── Dockerfile             # FROM ghcr.io/umami-software/umami:postgresql-v2.18.1
+├── docker-compose.yml     # postgres-15 + umami v2 on a fixed subnet, env-driven
+├── flow.sh                # bootstrap | record-traffic | coverage | list-routes
+├── keploy.yml.template    # globalNoise for createdAt/updatedAt/Date/uuid id fields
+└── README.md              # this file
+```
+
+## Contract
+
+The sample is keploy-independent: `docker compose up && bash flow.sh bootstrap && bash flow.sh record-traffic` runs end-to-end against bare umami. Lane scripts wrap that exact same path inside `keploy record` / `keploy test`.
+
+* `bootstrap` — log in as admin via `/api/auth/login`, capture the JWT-style auth token, persist it to `/tmp/umami-token-${UMAMI_PHASE}` so subsequent calls share a deterministic Authorization header.
+* `record-traffic` — drive the umami v2 API. Every call is logged to `${UMAMI_FIRED_ROUTES_FILE}` (when set) so the `coverage` subcommand has a numerator without needing a keploy recording. Calls are fire-and-forget (`|| true` semantics) so a single endpoint regression in umami itself does not abort the run — keploy is the assertion layer at replay.
+* `coverage` — walks the running container's `src/app/api/**/route.ts` tree as the denominator (the umami router is file-system based), compares against fired/recorded routes, emits a `(method, path)` percentage.
+* `list-routes` — diagnostic; prints the route table.
+
+## Local run
+
+```sh
+docker compose up -d
+bash flow.sh bootstrap 240
+UMAMI_FIRED_ROUTES_FILE=/tmp/fired.log bash flow.sh record-traffic
+UMAMI_FIRED_ROUTES_FILE=/tmp/fired.log bash flow.sh coverage
+docker compose down -v
+```
+
+## Consumers
+
+Lanes pinned to this sample:
+
+* `keploy/enterprise` `.woodpecker/umami-linux.yml` — record/replay matrix delegates compose + bootstrap + traffic to this sample.
+* `keploy/integrations` may add a `.woodpecker/umami-postgres.yml` falsifying lane in a future PR.