Skip to content

feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s - stacked PR (#2964) - #2965

Open
biswapanda wants to merge 30 commits into
PrimeIntellect-ai:mainfrom
biswapanda:bis/dyn-k8s
Open

feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s - stacked PR (#2964)#2965
biswapanda wants to merge 30 commits into
PrimeIntellect-ai:mainfrom
biswapanda:bis/dyn-k8s

Conversation

@biswapanda

@biswapanda biswapanda commented Jul 6, 2026

Copy link
Copy Markdown

Summary

Adds the Kubernetes DynamoGraphDeployment layer for the Prime RL Dynamo integration in #2964.

This branch is rebased on canonical Prime RL main at 84b413fd73a5b5f85f772e42bf1beb52fe0a265e, stacks the core branch at f077fbf9620f144ec2cd5d74c4dc0eb147f5ceb2, and has deployment head be5b75cfd8fbbc3f86415603cd773629473a7282.

What changed

  • renders Dynamo disaggregated prefill/decode topology as a DynamoGraphDeployment;
  • realizes PVC claims as concrete pod volumes and mounts;
  • supports direct GPU resource scheduling where DRA is unavailable;
  • creates the RL discovery service and selects the Grove-managed frontend deterministically;
  • records immutable config, topology, workload, manifest, commit, and image-digest provenance;
  • adds Helm/DGD integrity and deployment coverage.

Why it is needed

The core provider can launch Dynamo locally, but cluster operation requires the same topology and administrative contract to survive Helm rendering, Grove pod creation, Kubernetes API defaulting, service selection, PVC mounting, and digest-pinned image deployment. These changes make that deployment deterministic and auditable.

Validation

  • 181/181 stacked Prime inference/Kubernetes tests passed;
  • 17/17 orchestrator/barrier tests passed;
  • 11/11 Dynamo configuration tests passed;
  • live native-Dynamo and OpenEngine Qwen3-30B-A3B-Thinking-2507 Hendrycks Math runs both passed:
    • one frontend, two TP1 prefill workers, two TP1 decode workers, and a four-GPU EP/FSDP trainer;
    • five rollout and trainer steps per mode with maximum off-policy distance 0;
    • all four workers reached weight version 3;
    • post-training generation returned 16 tokens;
    • no surviving trainer/GPU processes;
    • both Helm releases were deleted and both artifact manifests verified.

Accepted evidence:

  • baseline: /data/openengine-e2e/runs/q30-baseline-final-20260711T111946Z
  • OpenEngine: /data/openengine-e2e/runs/q30-openengine-final-20260711T114759Z
  • exact image: nvcr.io/nvidian/dynamo-dev/biswa:prime-49aaf8bac3ce-dynamo-dce86a87a96b-vllm-b735dff3c15f-openengine-arm64@sha256:9c0f57ca6634a569e1df53089cecf6f5d6b622e9e49bb58ebc609f976157e1e6

@biswapanda biswapanda changed the title [DYNAMO] add DynamoGraphDeployment Helm mode feat(dynamo) add DynamoGraphDeployment Helm deployment for k8s Jul 7, 2026
@biswapanda biswapanda changed the title feat(dynamo) add DynamoGraphDeployment Helm deployment for k8s feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s Jul 7, 2026
@biswapanda
biswapanda marked this pull request as ready for review July 7, 2026 00:40
@biswapanda biswapanda changed the title feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s feat(dynamo): add DynamoGraphDeployment Helm deployment for k8s - stacked PR (#2964) Jul 7, 2026
Comment thread src/prime_rl/entrypoints/inference.py

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 7a2f9e7. Configure here.

Comment thread k8s/prime-rl/templates/deployment.yaml
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Signed-off-by: Biswa Panda <biswa.panda@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant