Skip to content

ci(e2e): stabilize product category catalog assertions - #4018

Draft
jczhuoMeta wants to merge 1 commit into
mainfrom
fix/product-category-e2e-stability
Draft

jczhuoMeta wants to merge 1 commit into
mainfrom
fix/product-category-e2e-stability

Conversation

@jczhuoMeta

Copy link
Copy Markdown
Contributor

Description

Stabilizes the product-category E2E shards after catalog writes were accepted but related products remained temporarily unavailable, or exposed stale product-set membership, during validation.

  • Drains pending WooCommerce product-sync jobs before remote validation and shares one in-flight drain across concurrent product checks.
  • Polls the exact expected state: the product must be fully synced and must contain, or no longer contain, the category's exact Meta product set.
  • Uses the category term_taxonomy_id retailer ID and compares Meta IDs as strings to avoid unsafe integer coercion.
  • Treats missing or malformed membership data as unknown, preventing category-deletion tests from passing when the product itself disappeared.
  • Gives the supported and latest category shards separate Meta asset lanes within a workflow run and rotates the selected lanes on reruns.
  • Replaces repeated whole-test replay with bounded condition polling, one Playwright retry, and an explicit job timeout.

This changes test infrastructure only. It does not modify plugin runtime behavior, WordPress user data, or database migrations.

Type of change

  • Fix (E2E test reliability)

Checklist

  • I have commented the non-obvious synchronization behavior.
  • No production PHP files are changed.
  • No user-data processing or database migration behavior is changed.
  • I followed general Pull Request best practices.
  • I added focused JavaScript tests and all JavaScript tests pass locally.
  • Full E2E validation against the shared Meta test assets is pending.
  • Documentation changes are not necessary because this affects test infrastructure only.

Changelog entry

N/A — test infrastructure only.

Test Plan

  • Run the complete JavaScript suite: 111 tests passed.
  • Run node --check on every changed JavaScript file.
  • Parse .github/workflows/product-creation-tests.yml as YAML.
  • Verify the three asset selectors remain distinct for the base, currently-supported category, and latest category lanes, including reruns.
  • Run git diff --check.
  • Run both GitHub product-category E2E shards against the real Meta test assets.

Expected CI behavior

Expected to pass:

  • E2E Tests - Currently supported - product-category
  • E2E Tests - Latest - product-category
  • JavaScript unit tests and static validation checks

Expected failure behavior:

  • If the local sync queue cannot be processed, the test fails immediately instead of polling unsent data.
  • If a product never reaches the exact expected product-set state, the test fails after the bounded polling deadline.
  • A missing product or incomplete product_sets response does not satisfy the category-deletion assertion.

Asset separation is intentionally scoped to contention within one workflow run. Concurrent workflow runs may still share the finite asset pool; exact state polling and unique generated fixtures remain the correctness safeguards in that case.

Screenshots

Not applicable.

@meta-cla meta-cla Bot added the CLA Signed label Aug 28, 2026
@github-actions github-actions Bot added the changelog: fix Took care of something that wasn't working. label Aug 28, 2026
@jczhuoMeta jczhuoMeta added the changelog: dev Developer-facing only change. label Aug 28, 2026
@github-actions github-actions Bot added the type: bug The issue is a confirmed bug. label Aug 28, 2026
@jczhuoMeta jczhuoMeta added type: technical debt This issue/PR represents/solves the technical debt of the project. and removed type: bug The issue is a confirmed bug. changelog: fix Took care of something that wasn't working. labels Aug 28, 2026
@github-actions

github-actions Bot commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

📦 Latest Plugin Build

Built at: 2026-11-05T10:26:13Z
Commit: ded63f1
Size: 1.9M

Download: Click here to download the plugin

To download: Click the link above → Scroll to bottom → Download "facebook-for-woocommerce" artifact

@codecov-commenter

codecov-commenter commented Aug 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@d44db31). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4018   +/-   ##
=======================================
  Coverage        ?   10.96%           
  Complexity      ?     5948           
=======================================
  Files           ?      227           
  Lines           ?    20264           
  Branches        ?        0           
=======================================
  Hits            ?     2222           
  Misses          ?    18042           
  Partials        ?        0           
Flag Coverage Δ
integration 10.96% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ukilla ukilla mentioned this pull request Sep 15, 2026
10 of 12 tasks
@vahidkay-meta
vahidkay-meta force-pushed the fix/product-category-e2e-stability branch from b9425cd to 34fd3d7 Compare November 5, 2026 10:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog: dev Developer-facing only change. CLA Signed type: technical debt This issue/PR represents/solves the technical debt of the project.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants