Skip to content

Let only the Test workflow save the shared uv cache - #207

Merged
ChristianGeng merged 2 commits into
mainfrom
fix/cache-suffix
Aug 6, 2026
Merged

Let only the Test workflow save the shared uv cache#207
ChristianGeng merged 2 commits into
mainfrom
fix/cache-suffix

Conversation

@ChristianGeng

@ChristianGeng ChristianGeng commented Aug 6, 2026

Copy link
Copy Markdown
Member

Problem

Every workflow in this repo installs uv via astral-sh/setup-uv,
and each one derives the same cache key — same runner OS, same Python version,
same dependency-file hash. When a single push triggers several workflows at
once, they all try to reserve that one key, and only the first to finish can
save it. The others log:

Failed to save: Unable to reserve cache with key setup-uv-2-x86_64-unknown-linux-gnu-ubuntu-24.04-3.12.13-5899ac3c8d62f4b9cac3f78a0d08699f23e7082de0032680922a422d775c3ad4, another job may be creating this cache.

Two live examples, both from the same push to main — note the identical
cache key in each, which is the collision itself:

Workflow Job Run
Documentation build (ubuntu-latest, 3.12) 30991884303
Linter build 30991884412

Both failed to reserve, so a third workflow from that same push won the race.

The warning is harmless in itself — the losing job's cache content would have
been byte-identical to the winner's, and nothing fails. It is pure log noise,
but it appears on most pushes and obscures annotations that do matter.

Fix

Keep the single shared cache key, but let only one workflow write it:
Documentation, Linter and Publish set save-cache: false on their setup-uv
step (the input is available in v9.0.0), while Test — the job with the fullest
environment — remains the sole saver. With one writer there is no reservation
race and no warning, cross-workflow cache sharing is preserved, and only a
single cache entry is stored.

Trade-off

After a cache-key rotation (e.g. a pyproject.toml change), Documentation and
Linter miss the cache until the next Test run saves the new key — a one-run
cold start costing roughly a second or two per job.

Context

This is a gap-closing change. This PR started as a per-workflow-cache draft
(cache-suffix: ${{ github.workflow }}) and pivoted to the single-saver
design after review feedback: per-workflow suffixes would have stored several
near-identical caches and given up cross-workflow sharing. audeer had
already merged #206 before the warning was diagnosed, so it
never received a fix. The same gap exists in opensmile-python, audformat
and audb, and each will get the same single-saver pattern in a follow-up PR.

Test plan

  • CI green
  • No Unable to reserve cache annotation on any job of this PR's runs

Every workflow installs uv via astral-sh/setup-uv, and each one derives
an identical cache key. When a single push triggers several of them
together, only the first to finish can reserve that key, and the rest
log "Failed to save: Unable to reserve cache with key ..., another job
may be creating this cache". Nothing breaks, since the losing job's
cache content would have been byte-identical to the winner's, but the
warning shows up on most pushes and buries the annotations that matter.

Setting cache-suffix to ${{ github.workflow }} gives each workflow its
own cache entry, so they stop contending for a shared one.

The trade-off is that workflows no longer share a warm cache with each
other, so each pays its own first-run download instead of one job
seeding it for the rest. The setup-uv step costs about a second, which
is negligible against a consistently clean log.

The repos handled later in the recent CI cleanup already received this
fix. audeer merged before the warning was diagnosed, so it never got
it; this closes that gap.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@sourcery-ai

sourcery-ai Bot commented Aug 6, 2026

Copy link
Copy Markdown
Contributor
Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

This PR updates all GitHub workflows that install uv so that each workflow uses a distinct cache key, eliminating cache reservation races between concurrently running workflows.

Sequence diagram for distinct uv cache per GitHub workflow

sequenceDiagram
    actor Developer
    participant GitHubActions
    participant doc_workflow
    participant linter_workflow
    participant uv_cache

    Developer->>GitHubActions: push to main
    GitHubActions->>doc_workflow: trigger doc.yml
    GitHubActions->>linter_workflow: trigger linter.yml

    doc_workflow->>uv_cache: setup-uv(cache-suffix=github.workflow)
    uv_cache-->>doc_workflow: reserve cache key setup-uv-2-...-doc

    linter_workflow->>uv_cache: setup-uv(cache-suffix=github.workflow)
    uv_cache-->>linter_workflow: reserve cache key setup-uv-2-...-linter

    doc_workflow->>doc_workflow: uv available from doc cache
    linter_workflow->>linter_workflow: uv available from linter cache
Loading

File-Level Changes

Change Details Files
Give each workflow its own uv cache by adding a workflow-specific cache suffix to the setup-uv step.
  • Add a with: block to each astral-sh/setup-uv step in all workflows.
  • Set cache-suffix to the current workflow name using the GitHub Actions context expression.
  • Ensure no other workflow logic or versions are changed in the process.
.github/workflows/doc.yml
.github/workflows/linter.yml
.github/workflows/publish.yml
.github/workflows/test.yml

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@ChristianGeng
ChristianGeng marked this pull request as ready for review August 6, 2026 08:24

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces GitHub Actions cache reservation races for astral-sh/setup-uv by ensuring each workflow uses a distinct cache key suffix, preventing multiple concurrently-triggered workflows from contending for the same uv cache entry.

Changes:

  • Add cache-suffix: ${{ github.workflow }} to the astral-sh/setup-uv@v9.0.0 step in all relevant workflows.
  • Ensure cache keys are workflow-scoped to eliminate “Unable to reserve cache” log noise during concurrent runs.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File Description
.github/workflows/test.yml Adds a workflow-specific setup-uv cache suffix to avoid cross-workflow cache key collisions.
.github/workflows/publish.yml Adds a workflow-specific setup-uv cache suffix to avoid cross-workflow cache key collisions.
.github/workflows/linter.yml Adds a workflow-specific setup-uv cache suffix to avoid cross-workflow cache key collisions.
.github/workflows/doc.yml Adds a workflow-specific setup-uv cache suffix to avoid cross-workflow cache key collisions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@ChristianGeng
ChristianGeng requested a review from hagenw August 6, 2026 08:30
@hagenw

hagenw commented Aug 6, 2026

Copy link
Copy Markdown
Member

If I understand it correctly the conflict only happens for updating the cache (re-uploading it at the end). All jobs should still be able to load the same cache, or are all the loosing jobs start with an empty cache?

In the proposed solution here we would store several caches even if they are more or less identical. Not sure if this is better.

The cache reservation race is save-time only and fires only when the
cache key rotates: losing jobs still restore the shared cache on
subsequent runs. Per review feedback, per-workflow cache suffixes
would store several near-identical cache entries and forfeit
cross-workflow sharing.

Instead, Documentation, Linter and Publish set save-cache: false
(input present in setup-uv v9.0.0) and Test — the job with the
fullest environment — remains the sole writer. No two writers means
no reservation race and no warning, while a single shared cache is
preserved.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@ChristianGeng ChristianGeng changed the title Give each workflow its own uv cache to stop reservation races Let only the Test workflow save the shared uv cache Aug 6, 2026
@ChristianGeng

ChristianGeng commented Aug 6, 2026

Copy link
Copy Markdown
Member Author
Correct — the conflict is save-time only. Losing jobs restore the shared cache
normally on later runs; nobody starts empty. And the warning only fires when the
cache key rotates (new dependency hash / first run with a new key): across the
last six pushes to `main` it appeared in 2 of 108 jobs, both on the #206 merge
that had just rotated the key — not on every push.

The storage point stands too: per-workflow suffixes would keep several
near-identical caches and give up cross-workflow sharing.

Pivoted in ef2738d: Documentation, Linter and Publish now set `save-cache:
false` (input available in setup-uv v9.0.0), and Test — the job with the fullest
environment — remains the sole writer. One writer means no reservation race and
no warning, while the single shared cache is kept.

The only cost: after a key rotation (e.g. a `pyproject.toml` change), the
non-saving workflows miss the cache once, until the next Test run saves the new
key — roughly a second or two per job for one run.

This writing above is too complicated. Now more bluntly:

The behavior now is that the setting is save-cache: false in

  • Documentation,
  • Linter
  • Publish

Test keeps the default and is the only writer. This also eliminates the race, as
now the writer is fixed by designation, not by finishing order.
always Test writes it - because it builds the fullest environment. With only one writer there is also nothing to race.

@hagenw

hagenw commented Aug 6, 2026

Copy link
Copy Markdown
Member

Thanks, the current approach makes sense.

In the single test cache, I assume the package versions for different Python versions are anyway stored at different locations and do not override each other?

@ChristianGeng

Copy link
Copy Markdown
Member Author

Thanks, the current approach makes sense.

In the single test cache, I assume the package versions for different Python versions are anyway stored at different locations and do not override each other?

Yes, they can't override each other, because the Python version is part of the cache key. There isn't one Test cache there's one per matrix entry:

  • …ubuntu-24.04-3.10.20-5899ac3c…
  • …ubuntu-24.04-3.11.15-5899ac3c…

and so on.

@ChristianGeng
ChristianGeng merged commit 91ce407 into main Aug 6, 2026
21 checks passed
@ChristianGeng
ChristianGeng deleted the fix/cache-suffix branch August 6, 2026 11:35
ChristianGeng added a commit to audeering/audformat that referenced this pull request Aug 6, 2026
* Let only Test save the shared uv cache

astral-sh/setup-uv derives its cache key from arch, runner, Python
version and the dependency hash, so every job running the same OS with
the same Python version lands on the same key. When one push starts
several such jobs, they all try to reserve that key at save time; the
first to finish wins and the rest log

    Failed to save: Unable to reserve cache with key setup-uv-2-...,
    another job may be creating this cache.

The failure is save-time only and harmless: the losing job would have
written identical content, and every job still restores the cache
normally. It is log noise.

Giving each workflow its own cache via cache-suffix would silence it
too, but that stores the same bytes several times over and was
rejected in audeering/audeer#207 in favour of a single saver.
Documentation, Linter and Publish therefore stop saving; Test keeps
the default and is the sole writer of the shared key.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Let one ubuntu-3.10 Test leg save the cache

The Test matrix declares ubuntu-latest with Python 3.10 twice -- once
plain, once with requirements: 'minimum'. Same OS and same Python
version means the same setup-uv cache key, so those two legs race with
each other inside Test, where the previous commit cannot reach them.

Measured twice on this repo, both times on a run whose keys were fresh:

  main 2e77f55, run 30999457855
    build (ubuntu-latest, 3.10)          saved 3.10.20
    build (ubuntu-latest, 3.10, minimum) Failed to save: Unable to
                                         reserve cache with key
                                         setup-uv-2-...-3.10.20-...

  PR #539 a577fa2, run 30996727386
    same pair, same key, same warning -- the loser was again the
    minimum leg

Order is not fixed: on the 2026-05-12 push to main the minimum leg won
and the plain leg lost.

The plain leg is the natural saver, since the minimum leg downgrades
several dependencies after uv sync, so let the variant stop saving.
GitHub renders the expression as true or false, which the boolean
input accepts.

Note that the run on this branch could not reproduce the warning: every
key was already present in the main scope, so setup-uv reported "Cache
hit occurred on key ..., not saving cache." and no job attempted a
save. The race only surfaces on the first run after a key rotation,
which is exactly what the two runs cited above were.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: cgeng <cgeng@audeering.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChristianGeng added a commit to audeering/audb that referenced this pull request Aug 6, 2026
* Let only Test save the shared uv cache

astral-sh/setup-uv derives its cache key from architecture, runner
image, Python version and a hash of the dependency files, so every job
that runs the same OS and the same Python version derives the same key.
When such jobs run from one push, they all try to reserve that key at
save time; the first to finish wins and the rest log

  Failed to save: Unable to reserve cache with key setup-uv-2-...,
  another job may be creating this cache.

The warning is save-time only and harmless -- the losing job would have
written identical content -- but it is log noise.

Giving each workflow its own cache via cache-suffix was considered and
rejected in audeering/audeer#207: it multiplies near-identical caches
for no benefit. Instead keep the one shared key and make exactly one
writer. Documentation, Linter and Publish set save-cache: false; Test
keeps the default and is the sole writer.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stop the two ubuntu 3.10 Test legs racing

The Test matrix declares ubuntu-latest + Python 3.10 twice, once plain
and once with pandas==2.1.4. Same OS and same Python means the same
setup-uv cache key, so those two legs race with each other and the
previous commit alone does not silence them.

Measured on this pull request with a cold key (run 31098374254,
attempt 2): both legs missed
setup-uv-2-x86_64-unknown-linux-gnu-ubuntu-24.04-3.10.20-ed955f29...,
the pandas leg saved it at 11:50:33 and the plain leg reported at
11:50:37

  Failed to save: Unable to reserve cache with key
  setup-uv-2-...-3.10.20-..., another job may be creating this cache.

That was the only warning left, and it came from this pair alone.

Making the variant leg skip the save leaves exactly one writer in Test:
matrix.pandas is unset for every other leg, so the expression is true
there and false for the pandas leg.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: cgeng <cgeng@audeering.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
ChristianGeng added a commit to audeering/opensmile-python that referenced this pull request Aug 6, 2026
astral-sh/setup-uv derives its cache key from the runner architecture,
the runner image, the Python version and a hash of the dependency
files. Documentation, Linter and the ubuntu-latest/3.10 leg of Test
therefore all derive the same key, and when one push triggers them
together they race to reserve it at save time. Only the first to finish
wins; the losers log

  Failed to save: Unable to reserve cache with key setup-uv-2-...,
  another job may be creating this cache.

The race is save-time only and harmless, since every loser would have
written identical content, but it is noise in an otherwise green log.

Giving each workflow its own cache-suffix would remove the collision by
storing several copies of the same cache, which was rejected as
wasteful in audeering/audeer#207. Keep the single shared key instead
and make exactly one workflow the writer: Documentation, Linter and
Publish set save-cache: false, so they still restore the cache but
never save it, and Test remains the sole saver.

Co-authored-by: cgeng <cgeng@audeering.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants