Skip to content

Adopt the "ArtifactResourceBudget" limiter from Pulpcore - #2553

Open
dralley wants to merge 1 commit into
pulp:mainfrom
dralley:resource-budget
Open

Adopt the "ArtifactResourceBudget" limiter from Pulpcore#2553
dralley wants to merge 1 commit into
pulp:mainfrom
dralley:resource-budget

Conversation

@dralley

@dralley dralley commented May 12, 2026

Copy link
Copy Markdown
Contributor

Puts a limiter on the amount of data that can be held locally between artifact download and saving (to content storage).

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

@dralley
dralley force-pushed the resource-budget branch 2 times, most recently from 5b89e4a to 0c325cf Compare May 13, 2026 21:04
@dralley
dralley marked this pull request as ready for review May 18, 2026 13:12
@dralley
dralley enabled auto-merge (rebase) May 18, 2026 16:04
Puts a limiter on the amount of data that can be held locally
between artifact download and saving (to content storage).
@dralley

dralley commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

Looks like we're missing some locking:

pulp [6e54e7dd-4c68-4b26-ab51-c411a32217b4]: pulpcore.deprecation:WARNING: Exception (OperationalError: deadlock detected
DETAIL: Process 2807 waits for ShareLock on transaction 3195; blocked by process 2612.
Process 2612 waits for ShareLock on transaction 3211; blocked by process 2807.
HINT: See server log for query details.
CONTEXT: while inserting index tuple (0,8) in relation "ansible_ansiblenamespace_pulp_domain_id_name_752ab148_uniq") will be sanitized in pulpcore 3.130
pulp [6e54e7dd-4c68-4b26-ab51-c411a32217b4]: pulpcore.tasking.tasks:INFO: Task[pulp_ansible.app.tasks.collections.sync] 019fa49a-f785-71cb-9c5b-22e5911ef48a failed (OperationalError: deadlock detected
DETAIL: Process 2807 waits for ShareLock on transaction 3195; blocked by process 2612.
Process 2612 waits for ShareLock on transaction 3211; blocked by process 2807.
HINT: See server log for query details.
CONTEXT: while inserting index tuple (0,8) in relation "ansible_ansiblenamespace_pulp_domain_id_name_752ab148_uniq") in domain: default
pulp [6e54e7dd-4c68-4b26-ab51-c411a32217b4]: pulpcore.tasking.tasks:INFO: File "/usr/local/lib/python3.12/site-packages/pulpcore/tasking/tasks.py", line 86, in _execute_task

https://github.com/pulp/pulp_ansible/actions/runs/30288613841/job/90052887453?pr=2553#step:18:376

That's unrelated to this PR but should be investigated

Comment thread pyproject.toml
"jsonschema>=4.9,<4.27",
"Pillow>=10.3,<13", # Semantic Versioning https://pillow.readthedocs.io/en/stable/releasenotes/versioning.html
"pulpcore>=3.105.13,<3.130",
"pulpcore>=3.111.0,<3.130",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is 3.111 supposed to become a supported pulpcore branch any time soon?

(Moving the LB on an unsupported branch is the only thing that makes me a bit cautious here.)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could move to 3.115 if you want - I don't think in the past we've made a policy out of not doing that though.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💥! It looks like we really 🦆 up on this.
StableOrderingFilter (technically a feature) was backported to 3.105 but not 3.111 or anything between those.
Do you think it would be reasonable to add this feature based on whether the import succeeded and leave this constraint alone?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How valuable is it to keep the constraint wide open vs. just bump the lower bound?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants