OSAC-4909: Reflect vault provisioning failures in tenant status by jira-autofix[bot] · Pull Request #879 · osac-project/osac

jira-autofix · 2026-09-10T12:45:55Z

Problem

When ensureVaultNamespace fails (e.g., OpenBAO is unreachable), the error propagated up to Run() which returned before calling Tenants/Update. This left the tenant stuck in PENDING or SYNCED state with no indication of failure, causing an infinite retry loop with no user-visible status change.

Fix

Modified ensureVaultNamespace to handle errors by:

Setting the tenant state to TENANT_STATE_FAILED with a descriptive message
Updating the VaultReady condition to FALSE with reason ProvisionFailed
Returning nil instead of the error, so the status update is persisted

Added early return guards after the ensureVaultNamespace call in both:

syncToIDP — prevents persistBreakGlassSecret and subsequent operations from overwriting the FAILED state with SYNCED
update (SYNCED path) — prevents checkDefaultNetworkingReadiness from running after a vault failure

This follows the same error-handling pattern used for CreateTenant failures.

Tests

Added two test cases to the "Vault namespace provisioning" suite:

Vault failure during initial sync (PENDING tenant)
Vault failure for an already-SYNCED tenant

openshift-ci-robot · 2026-09-10T12:45:59Z

@jira-autofix[bot]: This pull request references OSAC-4909 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the bug to target the "5.1.0" version, but no target version was set.

Details

In response to this:

Problem

When ensureVaultNamespace fails (e.g., OpenBAO is unreachable), the error propagated up to Run() which returned before calling Tenants/Update. This left the tenant stuck in PENDING or SYNCED state with no indication of failure, causing an infinite retry loop with no user-visible status change.

Fix

Modified ensureVaultNamespace to handle errors by:

Setting the tenant state to TENANT_STATE_FAILED with a descriptive message

Updating the VaultReady condition to FALSE with reason ProvisionFailed

Returning nil instead of the error, so the status update is persisted

Added an early return guard in syncToIDP after the ensureVaultNamespace call to prevent subsequent code from overwriting the FAILED state with SYNCED.

This follows the same error-handling pattern used for CreateTenant failures.

Tests

Added two test cases to the "Vault namespace provisioning" suite:

Vault failure during initial sync (PENDING tenant)

Vault failure for an already-SYNCED tenant

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

github-actions · 2026-09-10T12:51:40Z

🧭 E2E Suite Selection (POC, informational only)

Suite	Decision	Source	Reason
VMAAS	regression	gemini	Changes tenant provisioning failure handling, which affects all resource types.
CAAS	regression	gemini	Changes tenant provisioning failure handling, which affects all resource types.
BMAAS	regression	gemini	Changes tenant provisioning failure handling, which affects all resource types.

AI judgment confidence: 100%. This comment is informational only; nothing is gated on it yet.
_{Estimated cost: $0.0062 (3380 input + 196 output tokens, gemini-2.5-pro)}

coderabbitai · 2026-09-10T15:43:11Z

Important

Review skipped

Bot user detected.

To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository: osac-project/coderabbit/.coderabbit.yaml
Review profile: CHILL
Plan: Advanced
Run ID: ca3372f6-0fb9-4a2f-aafc-29ed8b4b3fae

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

🔍 Trigger review

_{Comment @coderabbitai help to get the list of available commands.}

eranco74 · 2026-09-14T08:24:20Z

/lgtm

github-actions · 2026-09-14T08:24:54Z

E2E on `lgtm`

Label lgtm applied — starting expensive e2e (PR run replay).

Started: 3/3

omer-vishlitzky · 2026-09-18T15:32:20Z

/e2e-ready

omer-vishlitzky · 2026-09-18T15:32:21Z

/ok-to-test

github-actions · 2026-09-18T15:32:43Z

Labeled ok-to-test. Re-ran 2 failed run(s).

osac-ai · 2026-09-18T15:32:46Z

⏳ E2E BMaaS Full Install -- Running

Follow along.

⏳ E2E VMaaS Full Install -- Running

Follow along.

⏳ E2E CaaS Full Install -- Running

Follow along.

_{Total AI diagnostic cost for this PR: $0.2572 (70015 input + 9768 output tokens across 3 diagnoses)}

github-actions · 2026-09-18T15:32:48Z

Labeled e2e-ready on 8a0295f. Starting expensive e2e (cleanup removes the label on next push).

github-actions · 2026-09-18T15:33:05Z

E2E on `e2e-ready`

Label e2e-ready applied — not starting a new full-install run.

Started: 0/3
Already active/green (skipped rerun): 3
Skipped gate invalidation (full-install already active or in-flight).

github-actions · 2026-09-18T15:36:31Z

🧭 Jobs Selection (informational only)

E2E Suites

Suite	Decision	Source	Reason
VMAAS	sanity	gemini-inconclusive	AI judgment was inconclusive for this suite
CAAS	sanity	gemini-inconclusive	AI judgment was inconclusive for this suite
BMAAS	sanity	gemini-inconclusive	AI judgment was inconclusive for this suite

AI judgment confidence: 80%.
_{Estimated cost: $0.0145 (3719 input + 589 output tokens, gemini-3.1-pro-preview)}

Unit Tests

Job	Decision	Reason
fulfillment-service	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-metering	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-metering/adapters	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-metering/schema	run	This workflow has no per-component scoping -- runs for any non-doc change

Integration Tests

Job	Decision	Reason
fulfillment-service	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-operator	run	This workflow has no per-component scoping -- runs for any non-doc change
bare-metal-fulfillment-operator	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-aap	run	This workflow has no per-component scoping -- runs for any non-doc change
osac-installer	run	This workflow has no per-component scoping -- runs for any non-doc change

Helm Lint

Job	Decision	Reason
osac-operator	skip	No changed files matched this job's path filter
bare-metal-fulfillment-operator	skip	No changed files matched this job's path filter
fulfillment-service	skip	No changed files matched this job's path filter
osac-aap	skip	No changed files matched this job's path filter
osac-csi-driver	skip	No changed files matched this job's path filter
osac-metering	skip	No changed files matched this job's path filter
osac-installer	skip	No dependent component chart changed

Checks & Builds

Job	Decision	Reason
Check generated code (proto)	skip	No changed files matched this job's path filter
fulfillment-service checks	run	Matches this job's path filter
Build container image (osac-operator)	skip	No changed files matched this job's path filter
Build container image (bare-metal-fulfillment-operator)	skip	No changed files matched this job's path filter
ansible-lint (osac-aap)	skip	No changed files matched this job's path filter
Darwin keychain tests	skip	No changed files matched this job's path filter

Every table above is informational only -- nothing here gates whether a job actually runs. The E2E Suites table can use AI judgment for ambiguous files; every other table is deterministic-only (no AI).

When ensureVaultNamespace fails (e.g. OpenBAO is unreachable), the error was propagated up causing Run() to return before persisting the tenant status update. This left the tenant stuck in PENDING/SYNCED with no indication of failure. Now ensureVaultNamespace sets the tenant state to FAILED with a descriptive message, updates the VaultReady condition to FALSE with reason ProvisionFailed, and returns nil so the status is persisted. This follows the same pattern used for CreateTenant failures. In syncToIDP, an early return after ensureVaultNamespace prevents subsequent code from overwriting the FAILED state with SYNCED. Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com> Signed-off-by: aipcc-bot <aipcc-bot@redhat.com>

Add missing FAILED-state guard after ensureVaultNamespace in the SYNCED tenant path of update(), consistent with the existing guards after updateIDP and in syncToIDP. Without this guard, checkDefaultNetworkingReadiness would run after a vault provisioning failure, making unnecessary API calls and potentially returning errors that mask the FAILED state. Assisted-by: Claude claude-opus-4-6 <noreply@anthropic.com>

github-actions · 2026-10-06T21:25:52Z

Removed ok-to-test label due to new commits. An org member must re-approve with /ok-to-test.

CrystalChun · 2026-10-06T21:26:06Z

/ok-to-test

CrystalChun · 2026-10-06T21:26:12Z

/approve
/lgtm

openshift-ci · 2026-10-06T21:26:18Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: CrystalChun, jira-autofix[bot]

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~fulfillment-service/OWNERS~~ [CrystalChun]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

github-actions · 2026-10-06T21:26:29Z

Labeled ok-to-test. Re-ran 3 failed run(s).

github-actions · 2026-10-06T21:26:36Z

E2E on `lgtm`

Label lgtm applied — not starting a new full-install run.

Started: 0/3
Already active/green (skipped rerun): 3
Skipped gate invalidation (full-install already active or in-flight).

openshift-ci · 2026-10-07T17:42:58Z

New changes are detected. LGTM label has been removed.

github-actions · 2026-10-07T17:43:05Z

Removed ok-to-test label due to new commits. An org member must re-approve with /ok-to-test.

CrystalChun · 2026-10-07T17:43:11Z

/e2e-ready

github-actions · 2026-10-07T17:43:38Z

Labeled e2e-ready on 09bdac9. Starting expensive e2e (cleanup removes the label on next push).

github-actions · 2026-10-07T17:43:58Z

E2E on `e2e-ready`

Label e2e-ready applied — not starting a new full-install run.

Started: 0/3
Already active/green (skipped rerun): 3
Skipped gate invalidation (full-install already active or in-flight).

CrystalChun · 2026-10-07T19:05:25Z

/close

superseded by #1498

openshift-ci · 2026-10-07T19:05:30Z

@CrystalChun: Closed this PR.

Details

In response to this:

/close

superseded by #1498

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

openshift-ci-robot added the jira/valid-reference label Sep 10, 2026

openshift-ci Bot requested review from omer-vishlitzky and rgolangh September 10, 2026 12:46

jira-autofix Bot force-pushed the autofix/osac-4909 branch from b320617 to 5f90594 Compare September 10, 2026 15:43

openshift-ci Bot assigned eranco74 Sep 14, 2026

openshift-ci Bot added the lgtm label Sep 14, 2026

jira-autofix Bot deployed to e2e-test September 14, 2026 08:25 Active

jira-autofix Bot deployed to e2e-test September 14, 2026 08:26 Active

omer-vishlitzky force-pushed the autofix/osac-4909 branch from 5f90594 to 8a0295f Compare September 18, 2026 15:32

openshift-ci Bot added ok-to-test and removed lgtm labels Sep 18, 2026

omer-vishlitzky deployed to e2e-test September 18, 2026 15:32 — with GitHub Actions Active

github-actions Bot added the e2e-ready label Sep 18, 2026

omer-vishlitzky had a problem deploying to e2e-test September 18, 2026 15:32 — with GitHub Actions Failure

omer-vishlitzky deployed to e2e-test September 18, 2026 15:32 — with GitHub Actions Active

aipcc-bot added 2 commits September 20, 2026 01:13

github-actions Bot removed the ok-to-test label Oct 6, 2026

openshift-ci Bot added the ok-to-test label Oct 6, 2026

CrystalChun deployed to e2e-test October 6, 2026 21:26 — with GitHub Actions Active

openshift-ci Bot assigned CrystalChun Oct 6, 2026

openshift-ci Bot added the lgtm label Oct 6, 2026

openshift-ci Bot added the approved label Oct 6, 2026

CrystalChun deployed to e2e-test October 6, 2026 21:26 — with GitHub Actions Active

CrystalChun deployed to e2e-test October 6, 2026 21:27 — with GitHub Actions Active

osac-ci-bot enabled auto-merge October 6, 2026 21:33

NO-ISSUE: Fix fmt in tenant_reconciler_function.go

09bdac9

openshift-ci Bot removed the lgtm label Oct 7, 2026

github-actions Bot removed the ok-to-test label Oct 7, 2026

CrystalChun deployed to e2e-test October 7, 2026 17:43 — with GitHub Actions Active

CrystalChun had a problem deploying to e2e-test October 7, 2026 17:43 — with GitHub Actions Failure

CrystalChun deployed to e2e-test October 7, 2026 17:43 — with GitHub Actions Active

osac-ci-bot disabled auto-merge October 7, 2026 17:48

openshift-ci Bot closed this Oct 7, 2026

Conversation

jira-autofix Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Tests

Uh oh!

openshift-ci-robot commented Sep 10, 2026 • edited by openshift-ci Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Fix

Tests

Uh oh!

github-actions Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧭 E2E Suite Selection (POC, informational only)

Uh oh!

coderabbitai Bot commented Sep 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

eranco74 commented Sep 14, 2026

Uh oh!

github-actions Bot commented Sep 14, 2026

E2E on lgtm

Uh oh!

omer-vishlitzky commented Sep 18, 2026

Uh oh!

omer-vishlitzky commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

osac-ai Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

E2E on e2e-ready

Uh oh!

github-actions Bot commented Sep 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🧭 Jobs Selection (informational only)

E2E Suites

Unit Tests

Integration Tests

Helm Lint

Checks & Builds

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

CrystalChun commented Oct 6, 2026

Uh oh!

CrystalChun commented Oct 6, 2026

Uh oh!

openshift-ci Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

Uh oh!

github-actions Bot commented Oct 6, 2026

E2E on lgtm

Uh oh!

openshift-ci Bot commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

CrystalChun commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026

Uh oh!

github-actions Bot commented Oct 7, 2026

E2E on e2e-ready

Uh oh!

CrystalChun commented Oct 7, 2026

Uh oh!

openshift-ci Bot commented Oct 7, 2026

Uh oh!

This branch was successfully deployed

Reviewers

Assignees

jira-autofix Bot commented Sep 10, 2026 •

edited

Loading

openshift-ci-robot commented Sep 10, 2026 •

edited by openshift-ci Bot

Loading

github-actions Bot commented Sep 10, 2026 •

edited

Loading

coderabbitai Bot commented Sep 10, 2026 •

edited

Loading

E2E on `lgtm`

osac-ai Bot commented Sep 18, 2026 •

edited

Loading

E2E on `e2e-ready`

github-actions Bot commented Sep 18, 2026 •

edited

Loading

E2E on `lgtm`

E2E on `e2e-ready`