diff --git a/.devcontainer/codespaces/README.md b/.devcontainer/codespaces/README.md new file mode 100644 index 0000000..b763370 --- /dev/null +++ b/.devcontainer/codespaces/README.md @@ -0,0 +1,189 @@ +# Running in GitHub Codespaces + +This folder holds a **Codespaces-specific** dev container config +([`devcontainer.json`](./devcontainer.json)). It is a **second, separate** config +from the local VS Code one at [`../devcontainer.json`](../devcontainer.json) and +leaves it untouched. Both reuse the same +[`../docker-compose.yml`](../docker-compose.yml) stack — **no base service +definitions were modified.** The only additions are this `codespaces/` folder. + +The stack is the template's standard one: **postgis, iqgeo, keycloak, redis, +pgadmin, rq-dashboard** (plus `openbao` and `centrifugo` in disabled profiles). +This config adds one extra container, the **`keycloak-tls`** nginx sidecar +(explained below), and the `docker-outside-of-docker` feature for stack +inspection. + +## Selecting this config when creating a Codespace + +The config a Codespace uses is fixed at creation and **cannot be changed +afterward**. When you create a Codespace you **must** pick this config: + +**Code ▸ Codespaces ▸ ⋯ ▸ New with options…** → in the **"Dev container +configuration"** dropdown choose **"Project for customer MyProject +(Codespaces)"**, not the default local config. + +**Confirm you're on the right config** once it's up: +```bash +env | grep MYW_EXT_BASE_URL # should show https://-80.app.github.dev, NOT localhost +``` +If it shows `localhost`, you picked the default config — delete the Codespace and +recreate with the correct one. + +## Required org-level secrets + +The platform images live in a **private Harbor registry** +(`harbor.delivery.iqgeo.cloud`). Codespaces cannot log in interactively, so it +authenticates using specially-named **Codespaces secrets**. Create these at the +**organization** level (Org Settings ▸ Secrets and variables ▸ Codespaces) and +grant them to this repository: + +| Secret name | Value | +| --- | --- | +| `HARBOR_CONTAINER_REGISTRY_SERVER` | `harbor.delivery.iqgeo.cloud` | +| `HARBOR_CONTAINER_REGISTRY_USER` | your Harbor username | +| `HARBOR_CONTAINER_REGISTRY_PASSWORD` | your Harbor **CLI secret** (from your Harbor user profile) | + +The `*_CONTAINER_REGISTRY_SERVER` / `_USER` / `_PASSWORD` naming is recognized by +Codespaces automatically — it runs `docker login` against that registry during +container build **and during prebuilds**. No login script, no credentials in the +repo. + +> The `HARBOR_*` prefix is arbitrary; only the `_CONTAINER_REGISTRY_*` suffixes +> matter. Keep all three consistent. + +**No other secrets are required.** DB password, Keycloak admin password, and the +OIDC client secret all have non-sensitive dev defaults baked into the Compose +file (`${VAR:-default}`). Do **not** commit a `.env`. + +## Prebuilds (caching the pull/build on push) + +Enable a prebuild so the expensive image pull/build is cached and Codespaces +start in seconds: + +1. Repo Settings ▸ Codespaces ▸ **Set up prebuild**. +2. Branch: `dev` (and/or `main`). +3. Dev container configuration: **`.devcontainer/codespaces/devcontainer.json`**. +4. Trigger: **On push**. + +What gets cached: the built `iqgeo` service image, the sibling images pulled when +Compose comes up during the prebuild, and the `onCreateCommand` result +(`myw_product fetch node_modules`, which runs inside the prebuild). Per-start work +(`git config safe.directory`, docker socket perms) is in `postStartCommand` and +runs every time, including resume-from-prebuild. + +## Architecture: PRIVATE Keycloak via the nginx TLS sidecar + +This is the key design point of this config: **Keycloak is never exposed on a +public port.** No port in this config should be made Public. + +In Codespaces the browser reaches each service at a per-port HTTPS URL like +`https://-.app.github.dev`, not `localhost`/`keycloak.local`. +OIDC login needs **two** things to agree on Keycloak's issuer URL: + +1. the **browser**, which redirects to Keycloak's login page, and +2. the **app server**, which does **server-side discovery** — it fetches + Keycloak's `.well-known/openid-configuration` to load the token-signing keys. + (This version of the OIDC client populates its key store via dynamic + discovery; skipping it fails ID-token verification with `Unknown issuer`.) + +Both must use the **same** issuer (`https://-8080.app.github.dev`), but +we do not want to make 8080 Public to satisfy the server-side fetch. The +`keycloak-tls` sidecar resolves this: + +- It is an **`nginx:alpine`** container that, on start, generates a self-signed + cert (`openssl req -x509 … -subj '/CN=keycloak'`) and runs + `nginx -g 'daemon off;'`. +- [`nginx-keycloak.conf`](./nginx-keycloak.conf) listens on **443 with TLS** and + reverse-proxies to the internal `keycloak:8080`, forwarding + `X-Forwarded-Proto: https` / `X-Forwarded-Host` so Keycloak (running + `KC_PROXY=edge`) builds correct absolute URLs. +- Crucially, the sidecar declares a **Docker network alias** equal to the + external Keycloak hostname + (`${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}`). So when + the **app server** resolves that hostname for its server-side discovery, Docker + DNS points it at **nginx inside the network** — not out to the public internet. + nginx terminates TLS and proxies to Keycloak. + +Meanwhile the **browser** uses the same hostname via the **Private** forwarded +port 8080: Codespaces authenticates the request at its edge and tunnels it to the +container. Both paths see the same issuer URL, so discovery and login agree — +**with no public port.** + +This relies on two settings that are **already in the template**: + +- `httpc_params.verify=false` in + [`../devserver_config/oidc/conf.json`](../devserver_config/oidc/conf.json), so + the app accepts the sidecar's self-signed cert during server-side discovery. +- `KC_PROXY=edge` on Keycloak (also set in + [`docker-compose.codespaces.yml`](./docker-compose.codespaces.yml)), so + Keycloak trusts the `X-Forwarded-*` headers from nginx. + +Because the back-channel goes through nginx and the browser goes through the +Private forwarded port, **ROPC, API tokens, internal service-to-service auth, and +interactive web login all work without exposing anything publicly** — the right +posture for both humans and headless/remote agents. + +> Forwarded ports are HTTPS, single-host. Server-to-server calls inside the +> Docker network (app → keycloak via the sidecar alias, app → postgis, etc.) are +> unaffected. + +## How to test the stack starts cleanly + +1. Push this branch; ensure the three Harbor secrets exist at the org level. +2. Create a Codespace on this branch, selecting the "…(Codespaces)" config. +3. Watch the creation log — it should pull from Harbor without an auth error, + build the `iqgeo` image, and bring the stack up. +4. You are working *inside* the `iqgeo` service container, so check services by + network (same hostnames the app uses): + ```bash + curl -s -o /dev/null -w "app: %{http_code}\n" http://localhost:8080 + pg_isready -h postgis -p 5432 + # discovery through the nginx sidecar (self-signed cert, so -k): + curl -sk -o /dev/null -w "keycloak: %{http_code}\n" \ + "https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN}/realms/iqgeo/.well-known/openid-configuration" + (exec 3<>/dev/tcp/redis/6379) && echo "redis: open" + ``` + The `docker-outside-of-docker` feature also gives you the Docker CLI, so you + can inspect the stack directly: + ```bash + docker compose -f .devcontainer/docker-compose.yml \ + -f .devcontainer/codespaces/docker-compose.codespaces.yml ps + ``` + All services should be `running` (including `keycloak-tls`). +5. Open the **Ports** tab → open the **IQGeo App** (port 80) URL. The app should + load, and OIDC login should complete without making any port Public. + +## Programmatic / agentic spin-up + +For remote agents or scripts, create Codespaces from the CLI instead of the UI. +Passing `--devcontainer-path` removes the "wrong config" trap entirely. + +Helper: [`new-codespace.sh`](./new-codespace.sh) creates the Codespace with the +right config + machine and blocks until the stack is healthy: +```bash +# requires: gh auth refresh -h github.com -s codespace +CS=$(.devcontainer/codespaces/new-codespace.sh agent-001) # prints codespace name +gh codespace ssh -c "$CS" -- 'cd /opt/iqgeo/platform/WebApps/myworldapp/modules && myw_product test' +gh codespace delete -c "$CS" +``` +Override `REPO` / `BRANCH` / `MACHINE` / `IDLE` via env vars. Under the hood it's +just `gh codespace create --devcontainer-path .devcontainer/codespaces/devcontainer.json …` +plus a readiness poll (`curl localhost:8080` + `pg_isready`). Set up a +**prebuild** (above) so each spin-up is fast instead of a cold start. + +## Recommended machine size + +**8-core / 16 GB** (`premiumLinux`). 4-core/8GB is the bare minimum and risky: +under Codespaces there is no host headroom, and the stack runs Postgres, +Keycloak (JVM), the IQGeo appserver, redis, pgAdmin, rq-dashboard and the nginx +sidecar. + +## Caveats + +- **Postgres data (`pgdata` volume) does not survive a full _rebuild_.** Named + Docker volumes persist across stop/start, but a "Rebuild container" recreates + them — you'll get a fresh DB (auto-rebuilt by the entrypoint when + `MYW_DB_UPGRADE=YES`). Keycloak realm config is re-imported on every start. +- **OpenBao is dev-mode / in-memory** (disabled profile by default). If you + enable it, it starts unsealed with a static root token and **loses all data on + every restart** — re-seed any secrets after a restart/rebuild. diff --git a/.devcontainer/codespaces/devcontainer.json b/.devcontainer/codespaces/devcontainer.json new file mode 100644 index 0000000..fb1cead --- /dev/null +++ b/.devcontainer/codespaces/devcontainer.json @@ -0,0 +1,86 @@ +{ + // Codespaces-specific dev container. + // + // This is a SECOND config, separate from ../devcontainer.json (the local + // VS Code flow). It reuses the same ../docker-compose.yml stack — it does + // NOT modify any service definitions — but tunes hooks, ports, registry + // auth, and Keycloak networking for GitHub Codespaces. + // + // Architecture: Compose-based dev container (no Docker-in-Docker). Codespaces + // brings the whole stack up on the host Docker daemon and drops you INTO the + // `iqgeo` service to develop, exactly like the local flow. Sibling services + // (postgis, keycloak, redis, pgadmin, rq-dashboard, plus the keycloak-tls + // nginx sidecar) are started via compose depends_on and the runServices list. + // + // Keycloak stays PRIVATE: the keycloak-tls nginx sidecar TLS-terminates the + // Codespace's forwarded 8080 host internally, so no port is ever made Public. + // See README.md for the full explanation. + "name": "Project for customer MyProject (Codespaces)", + + // Paths are relative to THIS file. The base compose file's own relative paths + // (build context, bind mounts like ../custom) resolve against the compose + // file's location (../), so they still point at the repo root. The second + // file is a Codespaces-only override that points the app/Keycloak env at the + // Codespace's external HTTPS URLs; it does not change any base service body. + "dockerComposeFile": ["../docker-compose.yml", "docker-compose.codespaces.yml"], + "service": "iqgeo", + + // Same default services as the local config, plus the keycloak-tls sidecar. + // depends_on pulls in postgis, keycloak and redis. + "runServices": ["iqgeo", "rq-dashboard", "pgadmin", "keycloak-tls"], + + "workspaceFolder": "/opt/iqgeo/platform/WebApps/myworldapp/modules", + "shutdownAction": "stopCompose", + + "features": { + // Same copilot-cli feature as the local config. + "ghcr.io/devcontainers/features/copilot-cli:latest": {}, + // Docker CLI inside the dev container, talking to the host daemon (NOT + // docker-in-docker). Lets you run `docker compose ... ps/logs` against + // the running stack from this container for inspection. + "ghcr.io/devcontainers/features/docker-outside-of-docker:1": {} + }, + + // Ports published by the compose services. Codespaces forwards these to + // HTTPS URLs on *.app.github.dev. Keycloak (8080) stays PRIVATE — see README. + "forwardPorts": [80, 8080, 8443, 5432, 6379, 9181, 8090], + "portsAttributes": { + "80": { "label": "IQGeo App", "onAutoForward": "notify" }, + "8080": { "label": "Keycloak (OIDC)", "onAutoForward": "silent" }, + "8443": { "label": "Keycloak (HTTPS)", "onAutoForward": "silent" }, + "5432": { "label": "PostgreSQL", "onAutoForward": "silent" }, + "6379": { "label": "Redis", "onAutoForward": "silent" }, + "9181": { "label": "RQ Dashboard", "onAutoForward": "silent" }, + "8090": { "label": "pgAdmin", "onAutoForward": "silent" } + }, + + "customizations": { + "vscode": { + "settings": { + "editor.formatOnSave": true + }, + "extensions": [ + "ms-python.python", + "esbenp.prettier-vscode", + "dbaeumer.vscode-eslint", + "eamodio.gitlens", + "IQGeo.iqgeo-utils-vscode" + ] + } + }, + + "remoteUser": "iqgeo", + "containerUser": "www-data", + + // onCreateCommand runs during the PREBUILD and is cached in the prebuild + // snapshot. Put slow, idempotent, network-bound setup here so a push-time + // prebuild absorbs the cost (node_modules fetch). + "onCreateCommand": "myw_product fetch node_modules", + + // postStartCommand runs on every start (including resume from prebuild). + // Also relax the host docker socket perms so the iqgeo/www-data user can use + // the Docker CLI (docker-outside-of-docker) — the socket is recreated each + // start, so this belongs here rather than in the image. Best-effort. + // We do NOT run `myw_product watch` here: it blocks Codespace startup. + "postStartCommand": "git config --global --add safe.directory /opt/iqgeo/platform/WebApps/myworldapp/modules && (sudo chmod 666 /var/run/docker.sock 2>/dev/null || true)" +} diff --git a/.devcontainer/codespaces/docker-compose.codespaces.yml b/.devcontainer/codespaces/docker-compose.codespaces.yml new file mode 100644 index 0000000..dffebe1 --- /dev/null +++ b/.devcontainer/codespaces/docker-compose.codespaces.yml @@ -0,0 +1,71 @@ +# Codespaces-only Compose override. +# +# Merged ON TOP of ../docker-compose.yml (see dockerComposeFile in +# devcontainer.json). It does NOT replace or edit the base service definitions — +# it only overrides the environment values that must point at the Codespace's +# external HTTPS URLs, and adds the keycloak-tls nginx sidecar. +# +# Codespaces forwards each published port to: +# https://${CODESPACE_NAME}-.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN} +# Both variables are present in the Codespace environment at `compose up` time. +# +# NOTE: intended only for the Codespaces dev container config. Running it locally +# (where CODESPACE_NAME is empty) would produce broken URLs. +# +# ── Keycloak stays PRIVATE (nginx TLS sidecar) ─────────────────────────────── +# Keycloak's forwarded port 8080 is never made Public. Instead: +# * The browser reaches Keycloak via the Private forwarded HTTPS URL +# (https://${CODESPACE_NAME}-8080.app.github.dev), which Codespaces +# authenticates and proxies to keycloak:8080 inside the network. +# * The app server's SERVER-SIDE OIDC discovery uses the SAME external URL, +# but the keycloak-tls sidecar advertises that hostname as a Docker network +# alias — so discovery resolves INTERNALLY to nginx, which TLS-terminates +# and proxies to keycloak:8080. No public port is required. +# This relies on httpc_params.verify=false (devserver_config/oidc/conf.json) so +# the self-signed cert is accepted, and KC_PROXY=edge (set below). + +services: + iqgeo: + environment: + # The app is always reached at its external URL (port 80). + IQGEO_HOST: ${CODESPACE_NAME}-80.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} + MYW_EXT_BASE_URL: https://${CODESPACE_NAME}-80.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} + # Keycloak back-channel URL the app uses (discovery + token). This is + # the external HTTPS URL, but it resolves internally to the + # keycloak-tls sidecar via the network alias declared below. + KEYCLOAK_URL: https://${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} + + keycloak: + environment: + # Keycloak advertises its external HTTPS frontend URL. Bare host, no + # port (-1), HTTPS forced; edge proxy mode trusts X-Forwarded-* from + # the nginx sidecar / Codespaces edge. + KC_HOSTNAME: ${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} + KC_HOSTNAME_PORT: -1 + KC_HOSTNAME_STRICT_HTTPS: true + KC_PROXY: edge + IQGEO_DOMAIN: https://${CODESPACE_NAME}-80.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} + + # nginx TLS sidecar that keeps Keycloak PRIVATE. It owns the external + # Keycloak hostname as a Docker network alias, so the app server's + # server-side OIDC discovery resolves that hostname to nginx (here) instead + # of going out to the public internet. nginx terminates TLS with a + # self-signed cert and proxies to keycloak:8080. + keycloak-tls: + image: nginx:alpine + restart: always + depends_on: + - keycloak + volumes: + - ./nginx-keycloak.conf:/etc/nginx/conf.d/default.conf:ro + command: + - sh + - -c + - > + openssl req -x509 -nodes -newkey rsa:2048 -days 3650 + -keyout /tmp/key.pem -out /tmp/cert.pem -subj '/CN=keycloak' + && exec nginx -g 'daemon off;' + networks: + default: + aliases: + - ${CODESPACE_NAME}-8080.${GITHUB_CODESPACES_PORT_FORWARDING_DOMAIN:-app.github.dev} diff --git a/.devcontainer/codespaces/new-codespace.sh b/.devcontainer/codespaces/new-codespace.sh new file mode 100755 index 0000000..ac56cc0 --- /dev/null +++ b/.devcontainer/codespaces/new-codespace.sh @@ -0,0 +1,48 @@ +#!/usr/bin/env bash +# +# Create a new Codespace for this repo using the Codespaces dev container config, +# then wait until the full stack is healthy. Intended for programmatic / agentic +# spin-up — run it on your machine or in CI (NOT inside a Codespace). +# +# Requires: gh CLI authenticated with the `codespace` scope +# (gh auth login / gh auth refresh -h github.com -s codespace). +# +# Keycloak is kept PRIVATE by the nginx TLS sidecar, so there are no manual +# post-create steps — backend, API, ROPC and interactive web login all work +# without exposing any public port. +# +# Usage: +# .devcontainer/codespaces/new-codespace.sh [display-name] +# Env overrides: +# REPO (default: IQGeo/utils-project-template) +# BRANCH (default: current branch, else dev) +# MACHINE (default: premiumLinux — 8-core/16GB; see `gh api .../machines`) +# IDLE (idle-timeout, default: 30m) +set -euo pipefail + +REPO="${REPO:-IQGeo/utils-project-template}" +BRANCH="${BRANCH:-$(git rev-parse --abbrev-ref HEAD 2>/dev/null || echo dev)}" +MACHINE="${MACHINE:-premiumLinux}" +IDLE="${IDLE:-30m}" +CONFIG=".devcontainer/codespaces/devcontainer.json" +DISPLAY_NAME="${1:-}" + +command -v gh >/dev/null || { echo "ERROR: gh CLI not found." >&2; exit 1; } + +args=(--repo "$REPO" --branch "$BRANCH" --devcontainer-path "$CONFIG" + --machine "$MACHINE" --idle-timeout "$IDLE") +[ -n "$DISPLAY_NAME" ] && args+=(--display-name "$DISPLAY_NAME") + +echo "Creating codespace: repo=$REPO branch=$BRANCH machine=$MACHINE config=$CONFIG" >&2 +CS="$(gh codespace create "${args[@]}")" +echo "Created codespace: $CS" >&2 + +echo "Waiting for the stack to become healthy (app + postgis)..." >&2 +gh codespace ssh -c "$CS" -- \ + 'until curl -sf http://localhost:8080 >/dev/null 2>&1 && pg_isready -h postgis -q 2>/dev/null; do sleep 5; done; echo READY' \ + >&2 + +echo "Codespace ready." >&2 +# Print the codespace name on stdout so callers can capture it: +# CS=$(.devcontainer/codespaces/new-codespace.sh agent-001) +echo "$CS" diff --git a/.devcontainer/codespaces/nginx-keycloak.conf b/.devcontainer/codespaces/nginx-keycloak.conf new file mode 100644 index 0000000..0fe7d97 --- /dev/null +++ b/.devcontainer/codespaces/nginx-keycloak.conf @@ -0,0 +1,21 @@ +# nginx TLS sidecar for keeping Keycloak PRIVATE in Codespaces. +# +# Terminates TLS with a self-signed cert (generated at container start) and +# reverse-proxies to the internal keycloak:8080. The app server's server-side +# OIDC discovery hits this nginx via the Docker network alias for the external +# Keycloak hostname, so no public port is needed. X-Forwarded-* headers let +# Keycloak (KC_PROXY=edge) build correct absolute URLs. +server { + listen 443 ssl; + + ssl_certificate /tmp/cert.pem; + ssl_certificate_key /tmp/key.pem; + + location / { + proxy_pass http://keycloak:8080; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-Proto https; + proxy_set_header X-Forwarded-Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + } +}