From 2b1d8f790a0e2babc5ca25454571a21eab54a6d3 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Wed, 20 May 2026 18:52:45 +0200 Subject: [PATCH 1/4] improve docker / appsec sections --- crowdsec/references/appsec/deploy.md | 2 +- crowdsec/references/install/docker.md | 86 ++++++++++++++++++++------- 2 files changed, 66 insertions(+), 22 deletions(-) diff --git a/crowdsec/references/appsec/deploy.md b/crowdsec/references/appsec/deploy.md index b992fc2..9ba2cf6 100644 --- a/crowdsec/references/appsec/deploy.md +++ b/crowdsec/references/appsec/deploy.md @@ -125,7 +125,7 @@ See also: [../configure/bouncers/web-servers.md](../configure/bouncers/web-serve | Env | What changes | |---|---| | **systemd / bare-metal** | The recipe above as-is. | -| **Docker / compose** | AppSec runs inside the crowdsec container. Expose port 7422 on the compose network so bouncer containers can reach it. The acquisition file is mounted from the host or baked into a customised image. `docker compose exec crowdsec cscli appsec-*` for management commands. | +| **Docker / compose** | AppSec runs inside the crowdsec container and must `listen_addr: 0.0.0.0:7422`. Bouncer containers reach it via the service name + internal port (`appsec_url: http://crowdsec:7422`), not the published port. The acquisition file is mounted from the host or baked into a customised image. `docker compose exec crowdsec cscli appsec-*` for management commands. **Containerized lua bouncers need a DNS `resolver` — see [../install/docker.md](../install/docker.md) § Bouncer key bootstrap.** | | **Kubernetes / Helm** | The official chart has `appsec.enabled: true` plus values for `appsec.listen_addr`, `appsec.config`, and a separate `appsec` Service. Bouncers target the AppSec Service DNS name. Required collections/rules can be listed in the chart's hub config. | ## Advanced shapes diff --git a/crowdsec/references/install/docker.md b/crowdsec/references/install/docker.md index 1979fb2..ae07e0e 100644 --- a/crowdsec/references/install/docker.md +++ b/crowdsec/references/install/docker.md @@ -49,22 +49,45 @@ labels: { type: syslog } source: file ``` -A path that exists on the host but isn't mounted reads **0 lines** silently — -verify with `docker exec crowdsec cscli metrics show acquisition`. To read -*other containers'* logs instead of files, use the built-in Docker datasource -(`datasource_docker` is compiled in) with `source: docker` and the docker -socket mounted — but that's a different acquisition shape; start with file -mounts. - -### 2. `COLLECTIONS` only applies to a *fresh* config volume - -`COLLECTIONS`/`PARSERS`/`SCENARIOS` env vars run on first boot **when -`/etc/crowdsec` is empty**. Because the compose above persists -`cs-config:/etc/crowdsec`, editing the env var later does nothing — the volume -already has a config. After first boot, manage the hub with -`docker exec crowdsec cscli collections install …` (and `docker compose restart` -or `cscli` reload), or recreate the config volume. Don't expect changing the -env var to retro-install. +A path that exists on the host but isn't mounted reads **0 lines** — the source +simply doesn't appear in the metrics (the agent log carries a +`No matching files for pattern ` warning, so it's not entirely silent). +Verify with `docker exec crowdsec cscli metrics show acquisition`. + +To read *other containers'* logs instead of host files, use the built-in Docker +datasource (`datasource_docker` is compiled in): mount the docker socket +(`/var/run/docker.sock:/var/run/docker.sock:ro`) and add an acquisition like + +```yaml +# acquis.d/docker.yaml +source: docker +container_name: + - web # or container_name_regexp / use_container_labels +labels: + type: nginx +``` + +(verified: crowdsec subscribes to Docker events, tails the container's +stdout/stderr, and scenarios fire on the containerized workload). The engine +runs as root so it can read the socket; a non-root container needs the `docker` +group via `GID`. + +### 2. Hub env vars install on *every* boot — but never *uninstall* + +`COLLECTIONS`/`PARSERS`/`SCENARIOS` (and the other hub env vars) are processed on +**every** container start, not just the first — the entrypoint runs +`cscli install` for each listed item each time (skipping items you've +tainted or made local). So **adding** an item and recreating the container +(`docker compose up -d`) *does* install it, even on a persisted +`cs-config:/etc/crowdsec` volume. (Verified: added `crowdsecurity/nginx` to +`COLLECTIONS`, recreated, it installed.) + +The catch is the other direction: **removing** an item from the env var does +**not** uninstall it. To remove, use the matching +`DISABLE_COLLECTIONS`/`DISABLE_PARSERS`/… env var, or +`docker exec crowdsec cscli collections remove …`. Post-boot you can manage the +hub directly with `docker exec crowdsec cscli collections install …` then +`docker compose restart`. ### 3. Port conflict with a host-installed engine @@ -81,8 +104,12 @@ or the published port reaches nothing. Verified with `7423:7422` + a host ### 5. Other env-in-container realities -- **`GID`**: set it so the container user can read mounted journald - sockets/group-readable logs; mismatch = 0 lines read despite a correct mount. +- **`GID`**: the official image runs the engine as **root**, so plain + bind-mounted log *files* are read regardless of `GID` (root bypasses group + perms — a `GID` mismatch does **not** silently zero out a file mount on the + default image). `GID` matters when you run the container as non-root, or for + group-restricted **sockets**: set it to the owning group (e.g. `docker` for + the `source: docker` socket, or the journald log group) or those reads fail. - **Time skew**: a container with a wrong clock fails CAPI TLS (`cscli capi status` errors). Containers normally inherit host time — only an issue with custom runtimes. @@ -96,11 +123,28 @@ or the published port reaches nothing. Verified with `7423:7422` + a host docker exec crowdsec cscli bouncers add my-bouncer -o raw ``` -Use that key in the bouncer's config (`api_url` → the mapped host port, e.g. -`http://:8081/`). For declarative bootstrap, the image also honours -`BOUNCER_KEY_` env vars; `cscli bouncers add` post-hoc is simplest for +Use that key in the bouncer's config. For declarative bootstrap, the image also +honours `BOUNCER_KEY_` env vars (verified — the named bouncer appears in +`cscli bouncers list` on boot); `cscli bouncers add` post-hoc is simplest for one-offs. +**Endpoint depends on where the bouncer runs:** + +- *On the host* → the **mapped** host ports: `api_url: http://:8081`, + `appsec_url: http://:7423`. +- *In a container on the same compose network* → the crowdsec **service name + + internal** ports: `api_url: http://crowdsec:8080`, + `appsec_url: http://crowdsec:7422` (the unmapped container ports — don't use + the published `8081`/`7423` from inside the network). Verified end-to-end with + allow 200 / block 403 through a containerized nginx bouncer. + +> **lua/OpenResty bouncers in a container need a DNS `resolver`.** The nginx lua +> bouncer resolves `crowdsec` via lua cosockets, which ignore the system +> resolver. Without an explicit `resolver 127.0.0.11;` (Docker's embedded DNS) +> in the nginx `http` context, every LAPI pull and AppSec check fails with +> `no resolver defined to resolve "crowdsec"` and the bouncer silently falls +> back. Add that directive (or pin a fixed IP) for any containerized lua bouncer. + ## Management & diagnostics Every `cscli` command works via `docker exec crowdsec cscli …`. The bundled From 11c10e201fd4a7d0027f344b8976d24befcec47c Mon Sep 17 00:00:00 2001 From: "Thibault \"bui\" Koechlin" Date: Wed, 20 May 2026 19:07:37 +0200 Subject: [PATCH 2/4] Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> --- crowdsec/references/install/docker.md | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/crowdsec/references/install/docker.md b/crowdsec/references/install/docker.md index ae07e0e..bdfdf20 100644 --- a/crowdsec/references/install/docker.md +++ b/crowdsec/references/install/docker.md @@ -55,8 +55,16 @@ simply doesn't appear in the metrics (the agent log carries a Verify with `docker exec crowdsec cscli metrics show acquisition`. To read *other containers'* logs instead of host files, use the built-in Docker -datasource (`datasource_docker` is compiled in): mount the docker socket -(`/var/run/docker.sock:/var/run/docker.sock:ro`) and add an acquisition like +datasource (`datasource_docker` is compiled in). This requires mounting the +Docker socket (`/var/run/docker.sock:/var/run/docker.sock:ro`) and adding an +acquisition like: + +> [!WARNING] +> Mounting `/var/run/docker.sock` gives the container root-equivalent control +> over the Docker host, even when mounted `:ro`. Only do this on trusted hosts. +> If possible, prefer lower-privilege alternatives such as mounting specific +> log files, using a least-privileged log shipper, or running CrowdSec outside +> Docker. ```yaml # acquis.d/docker.yaml From 7f2a0b40651a45d1da0c5e64eda6e6e68b66f819 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Thu, 21 May 2026 09:09:46 +0200 Subject: [PATCH 3/4] fix docker inconsistency --- crowdsec/references/install/docker.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/crowdsec/references/install/docker.md b/crowdsec/references/install/docker.md index bdfdf20..317ecf1 100644 --- a/crowdsec/references/install/docker.md +++ b/crowdsec/references/install/docker.md @@ -30,8 +30,9 @@ volumes: ``` Bring up: `docker compose up -d`. The image installs the `COLLECTIONS` on -first boot (verified: `sshd`, `appsec-virtual-patching`, `appsec-generic-rules` -all `enabled` after startup). +startup — and re-runs the install on **every** start, not just the first (see +§2). Verified: `sshd`, `appsec-virtual-patching`, `appsec-generic-rules` all +`enabled` after startup. ## The gotchas that actually bite From f76959073f854c856f89587dd0043406a48fd380 Mon Sep 17 00:00:00 2001 From: Thibault Koechlin Date: Thu, 21 May 2026 09:12:02 +0200 Subject: [PATCH 4/4] deal with non root users --- crowdsec/SKILL.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/crowdsec/SKILL.md b/crowdsec/SKILL.md index 9840f51..57e0154 100644 --- a/crowdsec/SKILL.md +++ b/crowdsec/SKILL.md @@ -39,6 +39,21 @@ kubectl get pods -A 2>/dev/null | grep -i crowdsec If nothing matches and the user reports CrowdSec is installed, ask where: a vendor appliance, a custom image, a binary in `/opt/`, or a remote host. Otherwise pivot to install: see [references/install/](./references/install/). +## Privileges — bare-metal / systemd prerequisite + +On bare-metal/systemd, `cscli` and `crowdsec` need **root** (they read +`/etc/crowdsec/`, the DB under `/var/lib/crowdsec/`, and control the systemd +unit). Before running anything that touches config or state, confirm the user +is **root or has sudo**: + +```bash +id -u # 0 = root; otherwise the user needs sudo +``` + +If they are neither root nor a sudoer, **stop and ask them to grant it** — don't +guess. Once confirmed, run bare-metal commands as root or prefixed with `sudo`. +Docker/k8s commands run inside the container/pod and do not need this. + ## Step 2 — Detect the intent | Cue from user | Go to | @@ -74,7 +89,7 @@ For anything debug-shaped, the first move is almost always: ## Step 3 — Universal `cscli` cheat sheet -These work in every environment. In docker/k8s prefix with `docker exec ` / `kubectl exec -n --`. +These work in every environment. On bare-metal/systemd, prefix with `sudo` (unless you are root) — see **Privileges** above. In docker/k8s prefix with `docker exec ` / `kubectl exec -n --` (which run as root inside the container/pod). | Purpose | Command | |---|---|