This project creates an k3d demo cluster. It comes with an interactive setup, which allows you to setup a Kubernetes cluster for demo and showcase purposes.
The documentation is available on GitHub Pages.
-
You should have installed k3d with it's dependencies on your system. See official installation guide: https://k3d.io/v5.8.3/#installation
- Min k3d version: v5.8.3
-
A Docker-compatible container engine that exposes a Docker socket (Docker Desktop, Colima, or Rancher Desktop). k3d talks to the engine through the Docker API.
- Rancher Desktop: select the
dockerd (moby)container engine, notcontainerd— thecontainerdengine provides no Docker socket and cannot run k3d.
- Rancher Desktop: select the
-
For the [Manual examples](#Manual examples) you should have installed HELM > 3.0. See official installation guide: https://helm.sh/docs/intro/install/
-
jq needs to be installed on your system. See official installation guide:https://stedolan.github.io/jq/download/
The creation of the cluster and a simple sample deployment from the local registry can be called up as follows:
bash create-sample.sh
# sudo rights are needed, cause the script adds an dummy entry for the registry to /etc/hosts
# you will be asked for your users passwordYou will be asked some questions about the cluster deployment, like number of nodes, Ingress ports and the CNI: Cilium (default) or Calico can be installed instead of the k3s default Flannel installation (Cilium and Calico are mutually exclusive). Traefik is disabled and Kong Gateway (Gateway API) is installed unconditionally as the cluster's sole ingress controller — it installs Gateway API CRDs, a GatewayClass, a Gateway, and Kong Ingress Controller via Helm.
If you want classic Ingress alongside Kong, the HAProxy showcase installs HAProxy Ingress Controller as a secondary IngressClass.
At least you have the option to deploy httpbin sample deployment, which is deployed to the namespace demo.
The container from https://kennethreitz.org/ is used here. The sample is exposed via an HTTPRoute on the Kong Gateway, and a NodePort is also exposed, to demonstrate this in k3d. A PVC is created and mounted to the httpbin container.
The httpbin demo is deployed from the local running container registry, just for demo purpose, to show the usage of a user defined registry with k3d.
Optional (K8s > 1.24 needed): Headlamp Kubernetes Dashboard can be deployed on your sample cluster. After successful deployment you can browse the dashboard. The necessary login token you can get with the following command: kubectl create token headlamp --namespace kube-system
After running this script you can visit the Demo HTTPBin Application by typing 127-0-0-1.nip.io:<Load-Balancer-Port> in your Browser. If you are using an other DEMO_DOMAIN you can use <Cluster-Name>.<DEMO_DOMAIN>:<Load-Balancer-Port> (e.q. demo.example.com:8080).
All prompts can be skipped with NON_INTERACTIVE=1. Every default can be overridden via environment variable:
NON_INTERACTIVE=1 CLUSTER_NAME=ci CALICO_FLAG=No bash create-sample.shAvailable variables (with defaults): CLUSTER_NAME (demo), SERVERS (1), AGENTS (1), HTTP_PORT (8080), HTTPS_PORT (8081), REGISTRY_PORT (5002), CILIUM_FLAG (Yes), CALICO_FLAG (No), DASHBOARD_FLAG (No), HTTPBIN_SAMPLE_FLAG (Yes), CAPI_FLAG (No). This mode is used by the CI smoke tests in .github/workflows/smoke.yml.
The default parameters look like this:
---
apiVersion: k3d.io/v1alpha4
kind: Simple
metadata:
name: demo
servers: 1
agents: 1
ports:
- port: 8080:80
nodeFilters:
- loadbalancer
- port: 8081:443
nodeFilters:
- loadbalancer
- port: <RANDOM PORT>:30001
nodeFilters:
- agents:*
- port: <RANDOM PORT>:30053/udp
nodeFilters:
- servers:*
- port: <RANDOM PORT>:30053/tcp
nodeFilters:
- servers:*
registries:
create:
name: registry.localhost
host: "0.0.0.0"
hostPort: "5002"
config: |
mirrors:
"registry.localhost":
endpoint:
- http://registry.localhost:5002
options:
k3d:
wait: true
There are some samples included, which are not deployed automatically. They are useful to demonstrate the usage of some commonly used Kubernetes tools.
Samples included under the examples folder:
- ExternalDNS - https://github.com/kubernetes-sigs/external-dns
- Installation is documented here README
- OpenBao https://github.com/openbao/openbao & cert-manager https://github.com/cert-manager/cert-manager
- Installation is documented here README
- External Secrets Operator (ESO) - https://github.com/external-secrets/external-secrets
- Installation is documented here README
- Kong API Gateway (Enterprise) - https://github.com/Kong/kong
- Installation is documented here README
- Kong Gateway Operator - https://docs.konghq.com/gateway-operator/latest/
- Installation is documented here README
- Kuma Service Mesh - https://github.com/kumahq/kuma
- Installation is documented here README
- Kyverno - https://github.com/kyverno/kyverno
- Installation is documented here README
- CloudNativePG - https://cloudnative-pg.io
- Installation is documented here README
- Calico NetworkPolicies - https://github.com/projectcalico/calico (requires a cluster created with
CALICO_FLAG=Yes)- Manal steps and demo are documented here README
- SeaweedFS (S3-compatible object store) - https://seaweedfs.com
- Installation is documented here README
- Velero (Backup & Restore) - https://velero.io
- Installation is documented here README
- HAProxy Ingress Controller - https://haproxy-ingress.github.io
- Installation is documented here README
This k3d cluster deployment uses nip.io DNS resolution for demo purposes. DNS names for registry and demo Ingresses are resolved to local IP. Example: registry.127-0-0-1.nip.io is resolved to static IP 127.0.0.1.
Caution: In some cases your network-router doesn't allow to resolve IPs of your own, or private IP address range. As a workaround you can change the ENV variable DEMO_DOMAIN in helpers.sh. This will automatically add the registry entry in your local /etc/hosts file. But it doesn't add any sample Ingress hostnames to your local /etc/hosts, you have to do this manually.