Skip to content

fix: replace Node.js pipe() with Web Streams pipeThrough() to fix NotReadableError flake#360

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 5 commits into
masterfrom
yoann/fix-notreadableerror-blob-race-condition
May 22, 2026
Merged

fix: replace Node.js pipe() with Web Streams pipeThrough() to fix NotReadableError flake#360
gh-worker-dd-mergequeue-cf854d[bot] merged 5 commits into
masterfrom
yoann/fix-notreadableerror-blob-race-condition

Conversation

@yoannmoinet
Copy link
Copy Markdown
Member

Summary

Fixes a long-standing flaky test failure (NotReadableError: The blob could not be read) in the unit test suite.

Root Cause

The error is a race condition in createRequestData() in packages/core/src/helpers/request.ts.

The chain of events:

  1. getData() builds FormData with fs.openAsBlob()-backed blobs (lazy file references)
  2. createRequestData() serialized the form through a Request, then used Node.js Readable.fromWeb().pipe(createGzip()) to compress it
  3. pipe() immediately calls .resume() on the source stream, scheduling actual blob reads via process.nextTick()
  4. In tests, nock intercepts the fetch() call and replies without consuming the request body — the async chain completes almost instantly
  5. The test finishes → cleanup runs → rm(rootDir) deletes the output directory
  6. process.nextTick() fires, tries to read the now-deleted file-backed blob → NotReadableError
  7. The unhandled stream error gets picked up by dd-trace, which misattributes it to the test.each registration context, making it appear as a flaky failure on a random test

Fix

Replace Readable.fromWeb().pipe(createGzip()) with the Web Streams pipeThrough(new CompressionStream('gzip')). Web Streams are lazy: the pipeline only starts reading when the output stream is actually consumed (i.e., when fetch reads the body). Since nock intercepts without reading the body, no blob I/O ever starts, eliminating the race.

This also simplifies the code by removing the Node.js stream and zlib imports in favor of the Web Streams CompressionStream (available since Node.js 18).

Changes

  • packages/core/src/helpers/request.ts: Replace Gzip | Readable with ReadableStream, use CompressionStream instead of createGzip, remove Node.js stream/zlib imports
  • packages/plugins/error-tracking/src/sourcemaps/sender.test.ts: Update readFully() helper to use Web Streams API to match the new ReadableStream type

Test plan

  • yarn test:unit packages/plugins/error-tracking — all 9 sender tests pass
  • yarn test:unit packages/core — all 11 request helper tests pass
  • Run the integration test suite a few times to confirm the flake is gone

🤖 Generated with Claude Code

yoannmoinet and others added 3 commits May 12, 2026 17:29
…equestData

Node.js pipe() immediately puts the source stream into flowing mode via
process.nextTick(), which starts reading file-backed blobs (from fs.openAsBlob)
in the background. When nock intercepts fetch and replies without consuming
the request body, the async chain completes and test cleanup deletes the output
directory before the scheduled blob read fires. This causes an unhandled
NotReadableError that dd-trace misattributes to the test.each registration
context, making it appear as a flaky test failure.

Replacing with Web Streams pipeThrough(new CompressionStream('gzip')) keeps
the pipeline lazy: reading only starts when the output stream is consumed,
so no I/O races with file cleanup.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Uses the native Web Streams DecompressionStream to decompress the
gzip output directly, eliminating the zlib import and the intermediate
readFully helper.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@yoannmoinet
Copy link
Copy Markdown
Member Author

@codex review
@cursor review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses a flaky NotReadableError in unit tests by changing request-body gzip compression from Node.js stream.pipe() (which eagerly starts reading) to Web Streams pipeThrough() (lazy until consumed), preventing races with test cleanup when the request body is never read.

Changes:

  • Updated createRequestData() to return a Web ReadableStream and perform gzip compression via CompressionStream('gzip') instead of Node Readable.pipe(createGzip()).
  • Updated the sourcemaps sender unit test to decompress and read the request body via Web Streams (DecompressionStream + Response.text()), matching the new stream type.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
packages/core/src/helpers/request.ts Switches request body serialization/compression to Web Streams and updates RequestData to use ReadableStream.
packages/plugins/error-tracking/src/sourcemaps/sender.test.ts Adjusts test helper logic to read/decompress the new Web Stream payload.
Comments suppressed due to low confidence (1)

packages/core/src/helpers/request.ts:89

  • Content-Encoding is being set to 'multipart/form-data' when zip is false. That value is not a valid content-encoding and can cause servers/proxies to mis-handle the request; the Request already provides the correct content-type (incl. boundary) via req.headers. Consider only setting Content-Encoding: gzip when zip is true, and otherwise omitting the header entirely (or, if you intended to set content type, set/leave content-type instead).
    const headers = {
        'Content-Encoding': zip ? 'gzip' : 'multipart/form-data',
        ...defaultHeaders,
        ...Object.fromEntries(req.headers.entries()),
    };

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@yoannmoinet yoannmoinet marked this pull request as ready for review May 13, 2026 21:17
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 00da4fb into master May 22, 2026
6 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the yoann/fix-notreadableerror-blob-race-condition branch May 22, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants