Skip to content

fix(drivers): a socket dying with a full write buffer no longer hangs the send (#131) - #132

Merged
productdevbook merged 1 commit into
mainfrom
fix/drain-hang
Sep 2, 2026
Merged

productdevbook merged 1 commit into
mainfrom
fix/drain-hang

Conversation

@productdevbook

Copy link
Copy Markdown
Owner

Closes #131. The last of the leads left open by the driver audit — and it turned out to be the worst defect in the set.

The hang

sendInternal() waited for backpressure to clear with a promise that had exactly one way to settle:

if (!socket.write(payload)) {
  await new Promise<void>((resolve) => socket.once("drain", () => resolve()))
}

If the peer vanished while the write buffer was still full, drain never fired and nothing else could reach that promise. failAll() only rejects waiters queued for a reply, and commandTimeoutMs belongs to those same waiters — neither covers this one.

Reproduced against a server that completes the handshake, stops reading once DATA begins so the client backs up, then destroys the socket:

P> HUNG — never settled

Four seconds is only where the probe gave up. The promise stays pending indefinitely.

Why it is the worst kind. A hang is not a failure, so nothing downstream can react to it: withRetry never gets control, and the pooled connection is never released. maxConnections slots leak one at a time until every acquire() queues behind a connection that will never come back, and the driver stops entirely. Getting there needs a message past the socket's 16 KB write buffer — any real attachment — and a connection that dies mid-transfer, which is an ordinary flaky-network event.

It is the same shape as #117 and the other half of it: a promise with a single settlement path and a peer that can disappear before it is taken. Both are now closed.

The fix races drain against error and close, removing all three listeners whichever wins, so a long-lived pooled connection does not accumulate a listener per write.

P> rejected: [unemail] [smtp] read ECONNRESET

STARTTLS listeners, as hardening

tls.connect({ socket }) leaves the pre-upgrade listeners on the socket underneath. Measured directly:

before wrap  data: 1 error: 1 close: 1
after  wrap  data: 1 error: 2 close: 2
raw still flowing (isPaused=false): true
our onData still attached: true

Anything the raw socket delivers after the upgrade is TLS records, which would go into the reply parser as noise, and the stale close handler would report the failure a second time. They are detached on upgrade now.

I could not settle whether those bytes actually reach the listener — that needs a TLS-capable fake server, and Node cannot mint a self-signed certificate without a dependency. So this is written up as hardening rather than as a fix for an observed failure. Removing listeners that should not be there cannot be wrong; claiming a bug I did not reproduce would be.

Verification

294 tests, including a regression test that drives the 4 MB stalling-server scenario end to end. Lint, format, typecheck, JSR dry run, build, bundle budgets and coverage floors all clean. connection.ts moved from 60% to 67% statements.

Still open

The remaining uncovered region of connection.ts is the STARTTLS upgrade path itself. Extending the fake server to speak TLS would close both that coverage gap and the question above — worth doing, and larger than this change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GQAPDWLnDrdYHKhou3Ntvb

… the send

Closes #131.

`sendInternal` waited for backpressure to clear with a promise that had one
way to settle: the `drain` event. If the peer vanished while the buffer was
still full, `drain` never fired and nothing else reached that promise —
`failAll` only rejects waiters queued for a reply, and `commandTimeoutMs`
belongs to those same waiters. `sendMessage()` never returned, the pooled
connection was never released, and `withRetry` could not help because
nothing had failed. Reproduced with a 4 MB message to a server that stops
reading and then closes: the promise stayed pending indefinitely. It now
races `drain` against `error` and `close`, removing all three listeners
whichever wins, so a pooled connection does not accumulate one per write.

Same shape as #117, and the second half of it: a promise with a single
settlement path and a peer that can disappear before it is taken.

While in there: `tls.connect({ socket })` leaves the pre-upgrade listeners
on the socket underneath — measured, the `data` handler is still attached
and the socket still flowing after wrapping. Anything delivered from there
would be TLS records going into the reply parser as noise, and the old close
handler would report the failure a second time. They are detached on
upgrade now. Whether those bytes actually arrive depends on Node internals I
could not exercise without a TLS-capable fake server, so this is written as
hardening rather than as a fix for an observed failure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01GQAPDWLnDrdYHKhou3Ntvb
@productdevbook
productdevbook merged commit 86faca5 into main Sep 2, 2026
3 checks passed
@productdevbook
productdevbook deleted the fix/drain-hang branch September 2, 2026 00:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(drivers): a socket that dies while the write buffer is full hangs sendMessage forever

1 participant