Skip to content

Attempt at fixing flakey tests #4700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hlipsig wants to merge 7 commits into
base: master
Choose a base branch
from
+13 −4
Open

Attempt at fixing flakey tests #4700

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
deccbff
Attempt at fixing flakey tests
hlipsig Mar 19, 2026
37bacd3
Initial plan
Copilot Mar 20, 2026
7543a0a
Revert flakey test changes and increase SpecTimeout to 3 minutes
Copilot Mar 20, 2026
cf97afa
Update .bingo/go.mod
hlipsig Mar 20, 2026
b8451e7
Merge pull request #4701 from Azure/copilot/sub-pr-4700
hlipsig Mar 20, 2026
01d2b1c
update both tests to use the eventually time out functions. Clear the…
hlipsig Apr 3, 2026
4332e70
remove unneeded context in func()
hlipsig Apr 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 13 additions & 4 deletions test/e2e/operator.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -446,8 +446,11 @@ var _ = Describe("ARO Operator - MUO Deployment", func() {
DeleteK8sObjectWithRetry(ctx, deleteFunc, managedUpgradeOperatorDeployment, metav1.DeleteOptions{})

By("waiting for the MUO deployment to be reconciled")
GetK8sObjectWithRetry(ctx, getFunc, managedUpgradeOperatorDeployment, metav1.GetOptions{})
}, SpecTimeout(2*time.Minute))
Eventually(func(g Gomega, ctx context.Context) {
_, err := getFunc(ctx, managedUpgradeOperatorDeployment, metav1.GetOptions{})
g.Expect(err).NotTo(HaveOccurred())
}).WithContext(ctx).WithTimeout(DefaultEventuallyTimeout).Should(Succeed())
Comment on lines +449 to +452
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Eventually block duplicates the existing GetK8sObjectWithRetry helper (used throughout this file) and also changes retry semantics (helper uses DefaultTimeout and PollingInterval, while this uses DefaultEventuallyTimeout and the suite’s default polling interval). To keep retries/timeouts consistent across the e2e suite and avoid future tuning in multiple places, prefer using the shared helper here (or explicitly match its timeout/polling settings).

Suggested change
Eventually(func(g Gomega, ctx context.Context) {
_, err := getFunc(ctx, managedUpgradeOperatorDeployment, metav1.GetOptions{})
g.Expect(err).NotTo(HaveOccurred())
}).WithContext(ctx).WithTimeout(DefaultEventuallyTimeout).Should(Succeed())
GetK8sObjectWithRetry(ctx, getFunc, managedUpgradeOperatorDeployment, metav1.GetOptions{})

Copilot uses AI. Check for mistakes.
})
Comment on lines 448 to +453
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This spec previously had a spec-level timeout and the PR description/linked context indicates the intent is to increase it to ~3 minutes to reduce flakes. The current change removes the SpecTimeout(...) entirely and instead relies on DefaultEventuallyTimeout (currently 5 minutes) for this wait, which doesn’t match the stated goal and changes the overall bounding behavior of the spec. Please reintroduce an explicit spec-level timeout (e.g., 3 minutes) or set this Eventually timeout to the intended value and keep a spec-level bound for the whole test.

Copilot uses AI. Check for mistakes.
})

var _ = Describe("ARO Operator - ImageConfig Reconciler", func() {
Expand Down Expand Up @@ -749,7 +752,10 @@ var _ = Describe("ARO Operator - Guardrails", func() {
DeleteK8sObjectWithRetry(ctx, deleteFunc, gkControllerManagerDeployment, metav1.DeleteOptions{})

By("waiting for the gatekeeper Controller Manager deployment to be reconciled")
GetK8sObjectWithRetry(ctx, getFunc, gkControllerManagerDeployment, metav1.GetOptions{})
Eventually(func(g Gomega, ctx context.Context) {
_, err := getFunc(ctx, gkControllerManagerDeployment, metav1.GetOptions{})
g.Expect(err).NotTo(HaveOccurred())
}).WithContext(ctx).WithTimeout(DefaultEventuallyTimeout).Should(Succeed())
Comment on lines +755 to +758
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reconciliation wait was changed from GetK8sObjectWithRetry to a custom Eventually with different timeout/polling defaults. Unless there’s a specific behavior change needed here, consider using the shared helper for consistency (or explicitly set polling/timeout to match it) to avoid subtly different retry behavior across similar tests.

Copilot uses AI. Check for mistakes.
})

It("Audit must be restored if deleted", func(ctx context.Context) {
Expand All @@ -771,7 +777,10 @@ var _ = Describe("ARO Operator - Guardrails", func() {
DeleteK8sObjectWithRetry(ctx, deleteFunc, gkAuditDeployment, metav1.DeleteOptions{})

By("waiting for the gatekeeper Audit deployment to be reconciled")
GetK8sObjectWithRetry(ctx, getFunc, gkAuditDeployment, metav1.GetOptions{})
Eventually(func(g Gomega, ctx context.Context) {
_, err := getFunc(ctx, gkAuditDeployment, metav1.GetOptions{})
g.Expect(err).NotTo(HaveOccurred())
}).WithContext(ctx).WithTimeout(DefaultEventuallyTimeout).Should(Succeed())
Comment on lines +780 to +783
Copy link

Copilot AI Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This reconciliation wait was changed from GetK8sObjectWithRetry to a custom Eventually with different timeout/polling defaults. Unless there’s a specific behavior change needed here, consider using the shared helper for consistency (or explicitly set polling/timeout to match it) to avoid subtly different retry behavior across similar tests.

Suggested change
Eventually(func(g Gomega, ctx context.Context) {
_, err := getFunc(ctx, gkAuditDeployment, metav1.GetOptions{})
g.Expect(err).NotTo(HaveOccurred())
}).WithContext(ctx).WithTimeout(DefaultEventuallyTimeout).Should(Succeed())
GetK8sObjectWithRetry(ctx, getFunc, gkAuditDeployment, metav1.GetOptions{})

Copilot uses AI. Check for mistakes.
})
})

Expand Down
Loading