feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s - stacked PR (#2964)

docs/training.md

@@ -58,7 +58,7 @@ A condensed view of the knobs you'll most often tune. For trainer-side paralleli
 |---|---|
 | `orchestrator.batch_size` | Tasks per trainer step. |
 | `orchestrator.group_size` | Rollouts generated per task. |
-| `orchestrator.max_off_policy_steps` | How many distinct policies may have contributed to one rollout before it's discarded (default 8). The main off-policy dial on long agentic rollouts — bump for throughput, lower for tighter on-policyness. Watch `errored_rollouts` and `mismatch_kl/all/mean` when tuning. |
+| `orchestrator.max_off_policy_steps` | On the vLLM admin backend, how many distinct policies may contribute to one rollout before it is discarded (default 8). Dynamo instead drains all live-policy/eval requests before mutating weights, so this setting does not apply to Dynamo runs. |
 | `[orchestrator.algo]` | Training algorithm — its `type` names it (`grpo` default, `max_rl`, `opd`, `opsd`, `sft`, `echo`). See [Algorithms](#algorithms). |
 | `[[orchestrator.train.env]]` | Training environments. List multiple tables for multi-env training; weight them via `ratio`. See [Configuration § Environments](configuration.md#environments-orchestratortrainenv). |
 | `[[orchestrator.eval.env]]` + `orchestrator.eval.interval` | Eval environments and cadence (default every 100 steps). |

k8s/README.md

@@ -59,6 +59,34 @@ helm install my-exp ./prime-rl \
   --set config.secrets.name=prime-rl-secrets
 ```
 
+### Generated DynamoGraph deployments
+
+`dynamo-dgd` produces internally consistent Helm values for Dynamo disaggregated inference. The source SHAs, image tag, and image digest are caller-supplied evidence: the renderer checks that they agree, but it does not authenticate source or image provenance. Verify those inputs through the release process before rendering and protect the generated values from modification.
+
+Chart-managed mode requires explicit runnable controller commands; the renderer supplies safe execution defaults of one replica and `autoStart: true` and refuses to emit a sleeper deployment:
+
+```bash
+uv run dynamo-dgd inference.toml \
+  --release-name my-exp \
+  --namespace my-namespace \
+  --image "$IMAGE@$IMAGE_DIGEST" \
+  --image-digest "$IMAGE_DIGEST" \
+  --prime-sha "$PRIME_SHA" \
+  --dynamo-sha "$DYNAMO_SHA" \
+  --output-dir ./artifacts \
+  --gpu-architecture arm64 \
+  --gpu-product NVIDIA-GB200 \
+  --gpu-node-pool prime-gpu \
+  --orchestrator-command 'uv run orchestrator @ /app/configs/debug/orch.toml --output-dir /data/outputs' \
+  --trainer-command 'uv run trainer @ /app/configs/debug/rl/train.toml --output-dir /data/outputs'
+
+helm install my-exp ./prime-rl \
+  --namespace my-namespace \
+  -f ./artifacts/dynamo-helm-values.json
+```
+
+Use `--external-controller` when another system runs orchestration and training; generated values then render no controller StatefulSets. Generated releases are limited to 41 characters so every derived Kubernetes Service name remains valid. The image tag must contain the first 12 characters of both caller-supplied source SHAs, and the digest must match `--image-digest`.
+
 ## Uninstalling
 
 ```bash

k8s/prime-rl/templates/_helpers.tpl

@@ -5,6 +5,42 @@ Expand the name of the chart.
 {{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
 {{- end }}
 
+{{/*
+Resolve the immutable image reference generated for DGD, with the native chart
+repository/tag remaining as the fallback for statefulset mode.
+*/}}
+{{- define "prime-rl.image" -}}
+{{- if .Values.image.reference -}}
+{{- .Values.image.reference -}}
+{{- else -}}
+{{- printf "%s:%s" .Values.image.repository .Values.image.tag -}}
+{{- end -}}
+{{- end }}
+
+{{/*
+Reuse a supplied shared claim or derive the chart-managed claim name.
+*/}}
+{{- define "prime-rl.storageClaimName" -}}
+{{- default (printf "%s-shared-data" .Release.Name) .Values.storage.existingClaim -}}
+{{- end }}
+
+{{- define "prime-rl.inferenceUrls" -}}
+{{- if eq .Values.inference.mode "dynamoGraph" -}}
+{{- printf "http://%s-frontend.%s.svc.cluster.local:8000/v1" .Release.Name .Values.namespace -}}
+{{- else -}}
+{{- $releaseName := .Release.Name -}}
+{{- $namespace := .Values.namespace -}}
+{{- $port := int .Values.inference.service.port -}}
+{{- $replicas := int .Values.inference.replicas -}}
+{{- $urls := list -}}
+{{- range $i := until $replicas -}}
+{{- $url := printf "http://%s-inference-%d.%s-inference-headless.%s.svc.cluster.local:%d/v1" $releaseName $i $releaseName $namespace $port -}}
+{{- $urls = append $urls $url -}}
+{{- end -}}
+{{- $urls | join "," -}}
+{{- end -}}
+{{- end }}
+
 {{/*
 Create a default fully qualified app name.
 */}}