Skip to content

Cache local mounts for filesystem access APIs - #6387

Merged
tonistiigi merged 3 commits into
moby:masterfrom
jedevc:cache-mounts-over-session
Jan 6, 2026
Merged

tonistiigi merged 3 commits into
moby:masterfrom
jedevc:cache-mounts-over-session

Conversation

@jedevc

@jedevc jedevc commented Nov 28, 2025

Copy link
Copy Markdown
Member

This PR adds optimizations for the issue described in #3037 (comment):

  • ReadFile/StatFile/ReadDir call mount+unmount for every single call.

Without this, any implementation of #3037 that directly accesses the root filesystem (instead of using the content API) is going to encounter serious performance issues. AFAICT, this has just always been an issue? But few frontends actually perform lots of filesystem accesses which would make this appear.

Somewhat inspired by @jsternberg's work in #6262, which would have added this API for gateway containers.

@jedevc jedevc changed the title Cache mounts over session Cache local mounts for filesystem access APIs Nov 28, 2025
@jedevc
jedevc requested a review from tonistiigi November 28, 2025 14:44
@tonistiigi
tonistiigi requested a review from jsternberg December 3, 2025 00:29
Comment thread frontend/gateway/forwarder/forward.go Outdated
return cacheutil.ReadDir(ctx, m, newReq)

var rd []*fstypes.Stat
err = withMount(m, func(root string) error {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why aren't these mounts cached the same way?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added them in a new commit. originally didn't add them, since they're really only used for building dockerfiles which don't actually use this operation a lot.

In addition to being wasteful, without this, concurrent calls to `Mount`
would have lost the previous `target` and so would never have been
removed.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Previously, each call to `ReadFile`/`ReadDir`/`StatFile` would trigger
an entire `Mount`+`Unmount` operation. This could result in terrible
performance for repeated filesystem accesses - potentially worse that
one `Mount`+`Unmount` per file, if the file is read in ranges.

To resolve this, we now directly store the `localMounter` in the
gateway, and rely on the fact that repeated calls to it return the same
value. These mounts will only be removed when the gateway itself is
released.

Signed-off-by: Justin Chadwell <me@jedevc.com>
Signed-off-by: Justin Chadwell <me@jedevc.com>
@jedevc
jedevc force-pushed the cache-mounts-over-session branch from 28e2d4d to 2485ab7 Compare December 4, 2025 11:33
@jedevc
jedevc requested a review from tonistiigi December 4, 2025 11:34
@tonistiigi
tonistiigi merged commit e799563 into moby:master Jan 6, 2026
169 of 170 checks passed
@jedevc
jedevc deleted the cache-mounts-over-session branch January 6, 2026 09:59
@crazy-max crazy-max added this to the v0.27.0 milestone Jan 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants