fix: run Cache Components staged renders correctly on workerd - #1318
fix: run Cache Components staged renders correctly on workerd#1318NathanDrake2406 wants to merge 14 commits into
Conversation
🦋 Changeset detectedLatest commit: 6b78393 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
commit: |
Dynamic partial prerenders could hang because Next\x27s atomic timer scheduler depends on private Node timer state that workerd does not expose. Cache interception could also finalize a cached PPR shell before Next resumed postponed work. Preserve staged render ordering with the unpatched immediate scheduler, and bypass interception only for partially prerendered routes so Next produces the complete stream. Cover dynamic params, warm caches, prefetches, and client navigation.
Turbo app-page runtimes strip createAtomicTimerGroup, causing the scheduler patch to silently no-op and Workers PPR requests to hang. Match structural timer mutation and fast-immediate markers across compiled runtimes and generated server chunks, and fail the build if the incompatible path survives.\n\nReal minified fixtures cover stable and canary output.
Application and dependency chunks can contain their own private timer feature checks. Identify Next's atomic timer group by its scheduling invariant and setTimeout structure so unrelated functions remain untouched while the incompatible scheduler still fails closed.
Next.js tracks in-flight dynamic imports on one CacheSignal per process. On Workers that signal holds immediate/timeout cleanup handles owned by whichever request scheduled them, so an overlapping request clearing them dies with "Cannot perform I/O on behalf of a different request" mid render and the isolate keeps serving truncated responses. Key the signal on the per-request Cloudflare context instead.
The scheduler and module loading signal matchers target code that ships in every Next 16.2+ bundle, and they fail the build when they stop matching. Gate them on the resolved Next config so apps without Cache Components cannot be broken by upstream reshaping code they never run.
Cache interception alone made the generated middleware bundle's shape build-critical for apps that never render Cache Components routes.
457bcbd to
69807b1
Compare
Next.js documents that an `import()` promise is commonly cached in user land, so only the first render executes the instrumented import and every later render learns about the pending promise from the shared signal. Per-request signals let a second render's `cacheReady()` resolve while a module it depends on was still loading. Keep the signal shared and drop only the request bound timer: the shared signal has no listeners to notify because `trackPendingModules` subscribes render signals instead, so the scheduled callback only ever walked empty arrays. Scheduling still happens when a listener exists, so a future Next.js that awaits this signal fails loudly rather than hanging. Also assert at build end that each patch actually matched, narrow the cache interception docs back to Cache Components, and check RSC responses against route content instead of a non-empty body.
|
Hi @NathanDrake2406! Could you give me an approximate timeline for when this is expected to be released? Thanks! |
|
Independent verification from a Next.js 16.3.1 / Cloudflare Workers app:
Full differential gate with the PR preview: concurrency 8, 100 requests per scenario, three scenarios (single 1 KiB boundary, single 64 KiB boundary, four parallel 64 KiB boundaries):
Every response was checked for HTTP 200, HTML content type, the static shell marker, every streamed boundary start/end marker, and minimum body length. This independently confirms that the scheduler patch in #1318 addresses the primary failure; payloads above 4 KiB are not required to reproduce it. |
Overlapping renders share Next.js module-loading CacheSignal state. Its cleanup closures retain request-owned timer handles, so later requests can fail with cross-request I/O and return truncated RSC streams. Keep only unresolved import promises in module scope. Seed a separate signal for each request, which preserves cached-import waiting without sharing timer handles. Strengthen concurrent response checks for incomplete HTML and Flight payloads.
Overlapping renders can subscribe before another request starts a cached import. A one-time pending-promise snapshot loses that future read and lets the render finish early. Forward new imports through request-owned promises, gate the rewrite to affected Next 16 releases, and fail closed without depending on the upstream backing identifier.
|
@kmsomebody can you re-test? |
|
Unfortunately this does not fix the issue. I did some debugging and noticed 2 things:
So it looks like both the cross-run interleaving and the scheduling/work performed during Hope this helps. |
Exhausting the settle budget entered the next stage anyway, which is the truncated response this scheduler exists to prevent, only silent. Reject instead so unsettled work surfaces as an error. That makes a leaked pending count fatal rather than merely slow, so keep the count honest on both sides of the pair: settle an immediate only once its clear has gone through, and drop the count again if the schedule threw.
that should do it |
|
I tested the latest commit and it seems that was not all that's necessary to fix this issue. The 1-byte responses are still there, but less frequent than before. I observed 30 prefetch requests for the same route, of which 11 had the 1-byte Interestingly, 2 of the 30 requests resulted in an incomplete response body of about 24 kB (uncompressed size). The remaining 17 requests had a 60 kB response body. In a local build, I consistently get the 60 kB response body. |
Next awaits the RSC payload before it starts staging, so React resumes much of the render from promises created outside `runInSequentialTasks`. Attributing immediates through an AsyncLocalStorage entered around the stage body never saw that work: the pending count read zero, every stage advanced over a flush still in flight, and the render slipped a stage - which reaches a runtime prefetch as a body holding only the partial marker. Next's own capture is process wide for exactly this reason. Charge immediates the run does not own to the Cloudflare request context, which wraps the whole render, and hold a stage until both counts settle. Requests still cannot gate each other's stages. Also trim the comments this PR added down to the repo's limit.
|
The issue remains with the latest commit, with approximately the same frequency as the previous commit. Potentially unrelated, but something I observed with this PR: export default defineCloudflareConfig({
incrementalCache: withRegionalCache(r2IncrementalCache, {
mode: "long-lived",
shouldLazilyUpdateOnCacheHit: true,
}),
queue: doQueue,
tagCache: doShardedTagCache({ baseShardSize: 12 }),
enableCacheInterception: false,
});In production we currently use opennextjs-cloudflare version I can test with version |
React can capture setImmediate before the scheduler installs, and timer groups from one request can interleave. Both paths can advance a stage before queued Flight work flushes, which truncates runtime prefetches or hangs the request. Install the scheduler before Next evaluates, and serialize staged groups within each request. Keep separate requests independent and release the queue before adopting an asynchronous render result. Add a large dynamic PPR fixture and stress cold, warm, overlapping, prefetch, and client-navigation paths with cache interception enabled or disabled.
|
I think it's fixed. Stressed it hard |
|
I'm still encountering the issues with the latest commit. |
Next captures the custom promise hook from setImmediate when it initializes node:timers/promises. Workerd does not provide this hook on its global callback API, so promise-based immediates throw and PPR requests fail. Capture the native promise API before Next initializes. Expose a counted hook that preserves values, cancellation, and staged settlement. Add a hostile PPR fixture with split imports, wide streams, prefetch overlap, and client navigation coverage.
|
Could you re-test the latest commit? |
|
It's still happening. I'll try to set up a reproduction repo tomorrow. |
|
Found a likely culprit while creating the reproduction repository. When I remove the Maybe related: getsentry/sentry-javascript#23592 |


Overview
cacheComponents: true) on Workers with the same output Node produces.CacheSignalrequest scoped, and let Cache Components routes bypass cache interception.500responses. Runtime prefetch payloads matchnext startbyte for byte. Apps without the flag are unaffected.Why
Next renders Cache Components as a pipeline of event loop tasks. Between two tasks, every immediate the previous task queued must run, so React flushes that stage before the next stage unblocks more content. A runtime prefetch aborts the render after its final task and discards every chunk that arrives later, so a flush that lands one task late is lost, not delayed.
Next builds that boundary from two Node behaviours:
createAtomicTimerGroupmutates the private_idleStartfield of timer handles so all stages share one timer phase.DANGEROUSLY_runPendingImmediatesAfterCurrentTaskdrains the captured immediates from aprocess.nextTick, which Node runs after the microtask queue is exhausted.Neither holds on workerd. Timer handles have no
_idleStart, so Next disables its own scheduling patch and logsNext.js cannot guarantee that Cache Components will run as expected.process.nextTickis implemented asqueueMicrotask:The drain therefore checks the immediate queue two microtask hops after the task starts. A React task that awaits even once has not scheduled its flush yet, so the queue looks empty, capturing stops, and every later flush becomes an ordinary macrotask behind the next stage.
Concrete case.
/deep-shell/[slug]in the experimental example renders a shell of nestedasynccomponents that each await a"use cache"read. On Next 16.3.1 the same request produces:next start)The same slip makes the document render fail. Next sees a
"use cache"read arrive in a later stage than expected and returns500withNext.js encountered uncached or runtime data during prerendering. In a deeper application the loss reaches the root, and the prefetch body is the single~partial marker with no Flight rows, which is what reviewers reported.runInSequentialTasksis its only consumer, so the replacement never engages it. Overlapping requests can no longer take the slot from each other.CacheSignaland its subscriptions are keyed on the Cloudflare request context.The scheduler works because workerd runs timers and immediates from one ordered macrotask queue and always drains microtasks between two of them. Scheduling an immediate is therefore an exact "everything queued before me has run" signal. Counting is attributed per render through
AsyncLocalStorage, so a busy isolate cannot make one render wait on another, and a hop cap keeps a render that never settles from stalling the pipeline.What changed
Next-Router-Prefetch: 2) of a deep shell~partial markernext start500 Internal Server Error200with the resolved shell and dynamic content500 Internal Server Error200with a Flight payloadCannot perform I/O on behalf of a different request, then empty or truncated bodies that outlive the requestenableCacheInterception: truecacheComponentsglobalThis.setImmediateis not wrappedMaintainer review path
packages/cloudflare/src/cli/templates/cache-components-scheduler.tsfor the replacement scheduler and why a macrotask hop is the correct boundary on workerd.packages/cloudflare/src/cli/build/patches/plugins/cache-components.tsforrunInSequentialTasksRule, which now delegates to that module, and for theonResolvehook that binds the generatedrequireto the copied template.packages/cloudflare/src/cli/templates/cache-components-scheduler.spec.tsfor the behavioural contract the scheduler must hold.examples/e2e/experimental/src/app/deep-shell/[slug]/page.tsxandexamples/e2e/experimental/e2e/staged-render.test.tsfor the regression fixture and its assertions.packages/cloudflare/src/cli/build/patches/plugins/cache-components.tsagain, forrequestScopedModuleLoadingSignalRuleandbypassPprCacheInterceptionRule, which are unchanged in intent from the earlier revisions of this PR.Validation
Regression coverage
cache-components-scheduler.spec.tsasserts that each stage's work and its flush land before the next stage, for microtask depths 0, 1, 3, and 12, and that two overlapping renders do not gate each other. Disabling the settle loop fails 5 of the 9 tests, so the suite is load bearing rather than tautological.cache-components.spec.tscovers the rewritten rule, the fixtures of the minified Turbopack schedulers fromnext@16.2.12and16.3.0-canary.105, and the resolution of the generatedrequireto the copied template.staged-render.test.tsasserts that runtime prefetches carry the shell they rendered, that the deep shell reaches the client, that the document and both prefetch variants return200, and that six overlapping runtime prefetches each keep their own session content.Differential check against Node
A Next 16.3.1 application was built once and served two ways:
next startandwrangler dev. The runtime prefetch of/deep-shell/[slug]matched at 5247 bytes with the fix, against 4773 bytes and a truncated shell without it.Load
Every response was checked for status, content type, the marker byte on runtime prefetches, the presence of Flight rows, the prerendered shell, and the resolved dynamic content.
The same harness against the previous revision of this branch reported 24 failures in 165 requests, all on
/deep-shell/[slug].Checks
pnpm code:checkspasses. The two affected vitest files pass (40 tests). The Playwright suite was not run to completion locally; the assertions instaged-render.test.tswere verified by issuing the same requests against the built worker.Scheduler trace, before and after
A harness running the real
fast-set-immediate.external.jsinside workerd, with a producer that models React (a gate resolves, the task awaits, then it schedules its flush throughsetImmediate):At depth 0, where the flush is scheduled synchronously from the gate, the previous revision behaved correctly. That is why the example application passed before this PR added a fixture with an awaiting shell.
Risk / compatibility
enableCacheInterceptionchanges, to record that Cache Components routes now bypass interception whileexperimental.ppralone still requiresfalse.globalThis.setImmediateandglobalThis.clearImmediateonce, lazily, on the first staged render. Only builds that enable Cache Components contain the module. When no staged render is active the wrapper delegates without bookkeeping.runInSequentialTasksis the only consumer ofDANGEROUSLY_runPendingImmediatesAfterCurrentTaskandexpectNoPendingImmediatesin Next's distribution. Replacing it leaves Next's fast-immediate capture inert rather than half used..open-next/cloudflare-templates/cache-components-scheduler.js, bundled into the server function.Non-goals
next@16.2.11onmainalready contains the matched code, and the checked-in fixtures cover16.2.12and16.3.0-canary.105.experimental.pprwith cache interception. That combination still requiresenableCacheInterception: false.References
app-render-render-utils.tsrunInSequentialTaskscontract this PR reimplementsapp-render-scheduling.tscreateAtomicTimerGroupand the_idleStartassumptionfast-set-immediate.external.tsprocess.nextTickorderinginternal_process.tsprocess.nextTickisqueueMicrotask~prefetch bodies and the interleaving observation that led to this revision