Skip to content

ci(bayn): add post-deploy production gate - #13392

Open
gregkonush wants to merge 14 commits into
mainfrom
codex/bayn-post-deploy-verify-p1-20260729
Open

ci(bayn): add post-deploy production gate#13392
gregkonush wants to merge 14 commits into
mainfrom
codex/bayn-post-deploy-verify-p1-20260729

Conversation

@gregkonush

@gregkonush gregkonush commented Jul 30, 2026

Copy link
Copy Markdown
Member

Summary

  • add a main-only Bayn post-deploy workflow scoped to merged deployment/kustomization promotions with GitHub contents: read permission
  • verify Argo promotion history/current-main lineage, exact source/tag/digest, Deployment/Pod convergence, restart state, /readyz, /v1/status, dependencies, autonomous loop, OBSERVE/read-only authority, zero mutations, and required reconciliation
  • fail closed with bounded retries, interruption handling, redacted errors, immutable-manifest validation, and runtime Kubernetes read/write permission checks
  • add deterministic regressions for the measured Argo-new-revision/old-image false positive and every required safety failure mode

Related Issues

None

Testing

  • bun test scripts/bayn-post-deploy-verify.test.ts — 77 passed, including core-versus-non-core wildcard RBAC parsing, wildcard/combined Secret cells, named Secret disclosure, skipped-intermediate Argo descendants, selective sync omission, bounded freshness, and deadline cancellation
  • bun node_modules/typescript/bin/tsc --ignoreConfig --noEmit --module esnext --moduleResolution bundler --target esnext --lib esnext,dom --types bun-types --skipLibCheck scripts/bayn-post-deploy-verify.ts scripts/bayn-post-deploy-verify.test.ts
  • bun node_modules/oxlint/bin/oxlint --config .oxlintrc.json scripts/bayn-post-deploy-verify.ts scripts/bayn-post-deploy-verify.test.ts — 0 warnings/errors
  • bun node_modules/oxfmt/bin/oxfmt --check .github/workflows/bayn-post-deploy-verify.yml scripts/bayn-post-deploy-verify.ts scripts/bayn-post-deploy-verify.test.ts
  • pre-commit lint-staged and commitlint hooks passed
  • root-script CI surface passed: Flamingo validation, Hermes production validation plus 130 tests, kube-router validation plus 24 tests
  • expanded Bayn CI surface passed locally: 926 service tests with 138 intentional skips, 136 verifier/package tests, type-aware lint, Effect diagnostics, TypeScript, and build
  • read-only live fresh verification passed against Argo 5e7af567a75841866e92f982095d51ac8abaa151, promoted source c7794ce4892ae7d9d6a7b38480a02fe1b39399b0, and digest sha256:13579bcffc30f6f2eaa4ba2054347b96888117236b286e27ac63374d9ea1db53
  • live end-to-end CLI permission preflight correctly failed closed because the current privileged operator identity has Argo write authority; the target arc-amd64 no-permission runner is read-only but currently lacks Bayn Pod reads and services/proxy, which is intentionally not changed in this PR because branch/environment/RBAC permissions are outside ownership
  • repository-wide Oxlint: 0 errors, 123 pre-existing warnings
  • repository-wide formatting remains blocked by pre-existing unrelated formatting drift and duplicate controllers keys in charts/agents/values-ci-ha.yaml

Breaking Changes

None. A future Bayn promotion will remain blocked with RBAC_DENIED until the existing read-only runner identity is separately granted only get/list access required for Bayn Pods and get services/proxy; this PR does not alter permissions.

Checklist

  • Testing section documents the exact validation performed.
  • Screenshots section removed; not applicable.
  • Breaking Changes and permission follow-up are documented.

@gregkonush

Copy link
Copy Markdown
Member Author

Reopening to retrigger repository pull-request checks; the initial opened event produced no Actions runs.

@gregkonush gregkonush closed this Jul 30, 2026
@gregkonush gregkonush reopened this Jul 30, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e35aa21372

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


jobs:
verify:
runs-on: arc-amd64

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Grant the selected runner its required cluster reads

The arc-amd64 scale set runs as arc-amd64-gha-rs-no-permission, but the bindings inspected in argocd/applications/agents-ci/runner-rbac-cluster.yaml grant that identity reads for Argo applications and deployments only; they do not grant the pod list or services/proxy get permissions required by validateReadOnlyPermissions. Consequently every triggered deployment gate exits with RBAC_DENIED before taking a snapshot, so add the Bayn-scoped read bindings or select an appropriately authorized runner identity.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed as a real deployment blocker. The selected arc-amd64-gha-rs-no-permission identity has read-only Argo/Deployment access but lacks Bayn Pod reads and services/proxy; the existing arc-arm64-gha-rs-kube-mode identity can read Pods/proxy but also has Argo mutation authority, so it does not satisfy the required minimum read-only contract. No existing runner identity satisfies all requirements. Adding the missing Bayn-scoped Role/RoleBinding or changing runner permissions is explicitly outside this task’s ownership (do not touch manifests ... branch/environment permissions). The verifier therefore correctly remains fail-closed with RBAC_DENIED; I am not marking this thread resolved or merging a gate that cannot run.

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head dd2b024. The Argo sync-resource shape finding is fixed. One prior P1 remains a documented scope blocker because no existing runner has the required minimum read-only RBAC.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd2b024700

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
Comment thread scripts/bayn-post-deploy-verify.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d598033019

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts
Comment thread scripts/bayn-post-deploy-verify.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: db755a128a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
Comment thread scripts/bayn-post-deploy-verify.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ac93d8063

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9444f1f54

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts
Comment thread scripts/bayn-post-deploy-verify.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c85bdc702e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
Comment thread scripts/bayn-post-deploy-verify.test.ts
@gregkonush
gregkonush force-pushed the codex/bayn-post-deploy-verify-p1-20260729 branch 2 times, most recently from f786279 to bd2602a Compare July 30, 2026 10:01

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bd2602ad6b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
@gregkonush
gregkonush force-pushed the codex/bayn-post-deploy-verify-p1-20260729 branch from 9d48196 to ef6a1fd Compare July 30, 2026 10:16

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ef6a1fdc85

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts
Comment thread scripts/bayn-post-deploy-verify.ts Outdated
@gregkonush
gregkonush force-pushed the codex/bayn-post-deploy-verify-p1-20260729 branch from ef6a1fd to cde25f0 Compare July 30, 2026 10:38

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cde25f048b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts
@gregkonush
gregkonush force-pushed the codex/bayn-post-deploy-verify-p1-20260729 branch from cde25f0 to 6831df1 Compare July 30, 2026 11:11
@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head 6831df1

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6831df170f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
Comment thread scripts/bayn-post-deploy-verify.ts Outdated
@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head cffa4e5

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cffa4e5044

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/bayn-post-deploy-verify.ts Outdated
@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head cffa4e5

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: cffa4e5044

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head fb4ad94

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: fb4ad94ffa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@gregkonush

Copy link
Copy Markdown
Member Author

@codex review exact head fb4ad94

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Nice work!

Reviewed commit: fb4ad94ffa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant