feat(gcpubsub): update existing subscriptions on config changes#2445
feat(gcpubsub): update existing subscriptions on config changes#2445rnagaraj13 wants to merge 6 commits intocelery:mainfrom
Conversation
When a subscription already exists, update it with current configuration instead of silently ignoring the AlreadyExists exception. This allows workers to automatically sync subscription settings (ack_deadline_seconds, expiration_policy, message_retention_duration, filters) without manual intervention in GCP Console. Fixes celery#2444
There was a problem hiding this comment.
Pull request overview
This pull request adds automatic synchronization of GCP Pub/Sub subscription settings when configuration changes are detected. Previously, when a subscription already existed, Kombu would silently ignore the AlreadyExists exception and not apply any configuration updates, requiring manual intervention in the GCP Console.
Changes:
- Modified
_create_subscriptionto update existing subscriptions with current configuration instead of silently ignoring them - Added comprehensive test coverage for the subscription update functionality including filter handling and graceful error recovery
- Moved subscription config creation outside the try block to enable reuse in both creation and update paths
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| kombu/transport/gcpubsub.py | Implements subscription update logic in the AlreadyExists exception handler, using Google Cloud's update_subscription API with appropriate FieldMask to sync ack_deadline_seconds, expiration_policy.ttl, message_retention_duration, and optional filter settings |
| t/unit/transport/test_gcpubsub.py | Adds comprehensive unit tests covering subscription updates when subscriptions exist, filter inclusion in update masks, and graceful handling of update failures |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #2445 +/- ##
==========================================
+ Coverage 81.19% 81.22% +0.02%
==========================================
Files 77 77
Lines 9738 9750 +12
Branches 1098 1099 +1
==========================================
+ Hits 7907 7919 +12
Misses 1625 1625
Partials 206 206 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
auvipy
left a comment
auvipy
left a comment
There was a problem hiding this comment.
please, fix the merge conflicts
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 2 out of 2 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hello, I opened a new PR with the code here rebased, just in case: #2532 |
When a subscription already exists, update it with the current configuration instead of silently ignoring the AlreadyExists exception. This allows workers to automatically sync subscription settings (ack_deadline_seconds, expiration_policy, message_retention_duration, enable_exactly_once_delivery, filters) without manual intervention in the GCP Console. Squashed rebase of #2445 (originally authored by ranjith nagaraj <rnagaraj13@users.noreply.github.com>) onto current main, with conflict resolution to preserve enable_exactly_once_delivery in both the create payload and the update mask. Fixes #2444 Co-authored-by: Asif Saif Uddin {"Auvi":"অভি"} <auvipy@gmail.com>
When a subscription already exists, update it with current configuration instead of silently ignoring the AlreadyExists exception. This allows workers to automatically sync subscription settings (ack_deadline_seconds, expiration_policy, message_retention_duration, filters) without manual intervention in GCP Console.
Fixes #2444