[1/3] Restrict Flex Node bootstrap token RBAC

docs/usages/aks-flex-config.md

@@ -9,7 +9,7 @@ The helper does not install anything on the target host. It uses Azure CLI and,
 - Azure CLI authenticated to the subscription that contains the AKS cluster.
 - `python3` on the workstation.
 - `kubectl` on the workstation for `setup-node-rbac` and `--bootstrap-token` config generation.
-- Permission to run `az aks get-credentials --admin` and create Kubernetes `ClusterRoleBinding` and bootstrap token `Secret` objects.
+- Permission to run `az aks get-credentials --admin`, create Kubernetes `ClusterRoleBinding` and bootstrap token `Secret` objects, and remove the obsolete `aks-flex-node-role` binding when present.
 
 ## Save The Helper
 
@@ -46,7 +46,53 @@ Run this once per cluster for bootstrap-token joins:
   --subscription "$SUBSCRIPTION_ID"
 ```
 
-This applies the bootstrap-related `ClusterRoleBinding` objects for the `system:bootstrappers:aks-flex-node` group.
+This applies only the CSR creation and approval `ClusterRoleBinding` objects for the `system:bootstrappers:aks-flex-node` group. If any binding still grants that group the obsolete `system:node` role, the command stops after applying the safe bindings and explains how to migrate. It does not silently remove the binding because older and development-mode agents may still use their bootstrap token after joining.
+
+`v0.1.1` introduced a separate daemon client certificate, but the version alone does not prove that certificate was issued successfully. Upgrade every bootstrap-token agent to `v0.1.1` or later (preferably the latest release), and on every host verify that the certificate exists, is unexpired, and the agent remains healthy after a restart:
+
+```bash
+sudo test -s /etc/aks-flex-node/daemon-credentials/daemon-controller-current.pem
+sudo openssl x509 \
+  -in /etc/aks-flex-node/daemon-credentials/daemon-controller-current.pem \
+  -noout -subject -enddate -checkend 0
+sudo systemctl restart aks-flex-node-agent.service
+sudo systemctl is-active aks-flex-node-agent.service
+```
+
+Then explicitly remove the obsolete binding:
+
+```bash
+./aks-flex-config setup-node-rbac \
+  --resource-group "$RESOURCE_GROUP" \
+  --cluster-name "$CLUSTER_NAME" \
+  --subscription "$SUBSCRIPTION_ID" \
+  --remove-legacy-node-role-binding
+```
+
+This migration is idempotent. It automatically deletes only the canonical `aks-flex-node-role` object created by older helpers. If another binding grants the same unsafe edge, or that object has extra subjects, the helper refuses to guess and identifies the objects for manual review. Bootstrap-token config generation refuses to create a token while any such binding exists, rather than either issuing an over-privileged token or unexpectedly breaking an old daemon.
+
+To verify no binding still grants the bootstrap group `system:node`, run:
+
+```bash
+kubectl get clusterrolebinding -o json | jq -r '
+  .items[]
+  | select(.roleRef.kind == "ClusterRole" and .roleRef.name == "system:node")
+  | .metadata.name as $binding
+  | .subjects[]?
+  | select(.kind == "Group" and .name == "system:bootstrappers:aks-flex-node")
+  | $binding'
+```
+
+The expected result is no output. The canonical `aks-flex-node-role` object is
+deleted; a safe, repurposed object with that name is preserved. Once certificate
+issuance has been verified, both the kubelet and long-running Flex daemon use
+issued client certificates, so removing the unsafe binding does not interrupt
+joined nodes. New and in-progress joins retain the CSR permissions installed
+above.
+
+Do not roll back a migrated host to an older or development-mode agent that still uses the bootstrap token for ordinary Kubernetes API requests. After this binding is removed, those requests correctly receive `403 Forbidden`. Restore a supported certificate-using agent instead of restoring the broad binding.
+
+Finally, delete bootstrap-token Secrets that are no longer needed. In particular, tokens made by helpers before `v0.1.1` had no expiration. Removing the broad binding limits them to bootstrap permissions, but does not revoke them; do not delete a token that is still being used by an in-progress join.
 
 ## Generate Node Config
 

docs/usages/joining-nodes.md

@@ -16,7 +16,7 @@ Bootstrap token mode is the recommended quickstart path. It uses Kubernetes TLS
 
 High-level flow:
 
-1. Run [`scripts/aks-flex-config setup-node-rbac`](../../scripts/aks-flex-config) to setup required node bootstrap RBAC permissions.
+1. Run [`scripts/aks-flex-config setup-node-rbac`](../../scripts/aks-flex-config) to set up the least-privilege node bootstrap RBAC permissions. Clusters configured by an older helper require the explicit compatibility migration documented in the helper guide before another token can be generated.
 2. Run `scripts/aks-flex-config generate-node-config --bootstrap-token` to create a bootstrap token, fetch AKS cluster metadata, and render the host config.
 3. Copy the generated config to `/etc/aks-flex-node/config.json` on the target host.
 4. Run `aks-flex-node preflight --config /etc/aks-flex-node/config.json` to validate host, cluster, rootfs, and artifact prerequisites without mutating the node.

hack/e2e/lib/node-join-kubeadm.sh

@@ -36,10 +36,10 @@ _kubeadm_ensure_rbac() {
   #  - ClusterRoleBindings for CSR creation and auto-approval
   #  - Roles/RoleBindings granting bootstrappers read access to kubeadm config
   #    and kubelet config (required by kubeadm join's preflight phase)
-  #  - ClusterRole/ClusterRoleBinding for bootstrappers to GET nodes
+  #  - ClusterRole/ClusterRoleBinding for kubeadm's bootstrap group to GET nodes
   #  - ConfigMaps: cluster-info (kube-public), kubeadm-config and
   #    kubelet-config (kube-system) consumed by kubeadm join
-  kubectl apply -f - <<EOF
+  if ! kubectl apply -f - <<EOF
 apiVersion: rbac.authorization.k8s.io/v1
 kind: ClusterRoleBinding
 metadata:
@@ -86,22 +86,6 @@ subjects:
   name: system:nodes
 ---
 apiVersion: rbac.authorization.k8s.io/v1
-kind: ClusterRoleBinding
-metadata:
-  name: aks-flex-node-role
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: ClusterRole
-  name: system:node
-subjects:
-- apiGroup: rbac.authorization.k8s.io
-  kind: Group
-  name: system:bootstrappers:aks-flex-node
-- apiGroup: rbac.authorization.k8s.io
-  kind: Group
-  name: ${kubeadmBootstrapGroup}
----
-apiVersion: rbac.authorization.k8s.io/v1
 kind: Role
 metadata:
   namespace: kube-system
@@ -175,18 +159,19 @@ roleRef:
   kind: ClusterRole
   name: kubeadm:get-nodes
 subjects:
-- kind: Group
-  apiGroup: rbac.authorization.k8s.io
-  name: system:bootstrappers:aks-flex-node
 - kind: Group
   apiGroup: rbac.authorization.k8s.io
   name: ${kubeadmBootstrapGroup}
 EOF
+  then
+    log_error "Failed to apply bootstrap RBAC"
+    return 1
+  fi
 
   # Publish the ConfigMaps that kubeadm join reads during its preflight phase.
   # cluster-info goes into kube-public (publicly readable).
   # kubeadm-config and kubelet-config go into kube-system (bootstrapper-readable).
-  kubectl apply -f - <<EOF
+  if ! kubectl apply -f - <<EOF
 apiVersion: v1
 kind: ConfigMap
 metadata:
@@ -229,6 +214,10 @@ data:
     apiVersion: kubelet.config.k8s.io/v1beta1
     kind: KubeletConfiguration
 EOF
+  then
+    log_error "Failed to apply kubeadm bootstrap ConfigMaps"
+    return 1
+  fi
 
   log_success "Bootstrap RBAC and ConfigMaps configured"
 }
@@ -307,10 +296,10 @@ node_join_kubeadm() {
   log_info "Creating bootstrap token..."
   local bootstrap_token
   bootstrap_token="$(with_cluster_lock _kubeadm_create_bootstrap_token)"
-  state_set "kubeadm_bootstrap_token" "${bootstrap_token}"
 
   # Step 2: Generate the config file for aks-flex-node agent
   local config_file="${E2E_WORK_DIR}/config-kubeadm.json"
+  install -m 0600 /dev/null "${config_file}"
   cat > "${config_file}" <<EOF
 {
   "azure": {