Skip to content

Filter benign unobserved-Task socket noise from Sentry (BL-16573)#8080

Open
hatton wants to merge 4 commits into
masterfrom
BL-16573-sentry-unobserved-socket-noise
Open

Filter benign unobserved-Task socket noise from Sentry (BL-16573)#8080
hatton wants to merge 4 commits into
masterfrom
BL-16573-sentry-unobserved-socket-noise

Conversation

@hatton

@hatton hatton commented Jul 19, 2026

Copy link
Copy Markdown
Member

[Claude Fable 5]

Screens out the benign "unobserved Task socket/IO abort" noise that the Sentry .NET SDK reports on our behalf. Three locale-split Sentry issues share one root cause and together account for ~71,000 events / 0 users over 90 days:

Root cause

The Sentry SDK auto-subscribes to TaskScheduler.UnobservedTaskException. Fleck (our WebSocket library, behind BloomWebSocketServer) does fire-and-forget socket.Send() calls whose faulted Tasks are never observed; when a socket aborts (mostly at app shutdown) the finalizer thread surfaces a benign SocketException/IOException and Sentry reports it. As Sentry serializes it, the event is an inner System.Net.Sockets.SocketException ("The I/O operation has been aborted...") wrapped in a System.AggregateException whose mechanism is UnobservedTaskException. An earlier IsAvailable-check mitigation (BL-11124) did not stop it.

Fix

Switch SentrySdk.Init to the Action<SentryOptions> overload (same DSN) and add a narrowly-scoped BeforeSend filter that drops ONLY this pattern: an event carrying the UnobservedTaskException mechanism whose exception chain contains a benign socket/IO abort type. The decision is carried by exception type + mechanism, never by message text, so it is locale-independent and covers all three issues (and future language variants) at once.

Genuine unobserved-Task bugs are still reported - we deliberately do NOT call DisableTaskUnobservedTaskExceptionCapture(). The filter predicate is a testable internal static method with unit tests in BloomTests covering a matching event (dropped), the localized-message (Spanish) variant, and non-matching events (passed through).

Ref: https://issues.bloomlibrary.org/youtrack/issue/BL-16573

🤖 Generated with Claude Code

Devin review


This change is Reviewable

The Sentry .NET SDK auto-subscribes to TaskScheduler.UnobservedTaskException.
Fleck (our WebSocket library, see BloomWebSocketServer) does fire-and-forget
socket.Send() calls whose faulted Tasks are never observed; when a socket
aborts (mostly at app shutdown) the finalizer thread surfaces a benign
SocketException/IOException and Sentry reports it. An earlier IsAvailable-check
mitigation (BL-11124) did not stop it, so ~71k events (0 users, 90d) piled up
across three locale-split Sentry issues: BLOOM-DESKTOP-EQ4 / -E4J / -E9K.

Switch SentrySdk.Init to the Action<SentryOptions> overload (same DSN) and add
a narrowly-scoped BeforeSend filter that drops ONLY this pattern: an event
carrying the UnobservedTaskException mechanism whose exception chain contains a
benign socket/IO abort type. The decision is carried by exception TYPE plus the
mechanism, never by message text, so it is locale-independent and covers all
three issues (and future language variants) at once. Genuine unobserved-Task
bugs are still reported; we deliberately do NOT disable the integration.

The filter predicate is a testable internal static method with unit tests in
BloomTests exercising matching (dropped), the localized-message variant, and
non-matching (passed through) events.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Jul 19, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds a narrowly-scoped BeforeSend Sentry filter that drops the benign "unobserved Task socket/IO abort" noise (three Sentry issues, ~71k events, 0 users) caused by Fleck's fire-and-forget socket.Send() calls surfacing as unobserved SocketException/IOException/cancellation at shutdown — without disabling the broader UnobservedTaskException integration.

  • IsBenignUnobservedTaskSocketNoise gates on both the UnobservedTaskException mechanism and the presence of only benign exception types, so genuine unobserved-Task bugs and mixed-fault AggregateExceptions are still reported.
  • Eight NUnit tests cover all four benign-type branches (drop), mixed chains containing a real bug (keep), the vacuous-All guard, mechanism-absent SocketException, and the empty-event edge case — including a parameterized case added post-initial-review for the two cancellation types.

Important Files Changed

Filename Overview
src/BloomExe/Program.cs Adds two well-commented helper methods for Sentry noise filtering and switches SentrySdk.Init to the options overload with a narrowly-scoped BeforeSend filter; logic is correct and safe-fails on unexpected shapes
src/BloomTests/ProgramTests.cs Adds 8 parameterized and standalone NUnit tests covering all four benign-type branches (drop), mixed chains with real bugs (keep), vacuous-truth guard, mechanism-absent case, and the empty-event edge case

Reviews (3): Last reviewed commit: "Merge remote-tracking branch 'origin/mas..." | Re-trigger Greptile

Comment thread src/BloomTests/ProgramTests.cs
Comment thread src/BloomExe/Program.cs
Comment thread src/BloomExe/Program.cs
…(BL-16573)

Greptile P2 feedback on the Sentry unobserved-task filter:
- Add coverage for the OperationCanceledException / TaskCanceledException
  branches of IsBenignSocketAbortExceptionType (they had no test).
- Explain in a code comment why IOException and the cancellation types are
  screened alongside SocketException, and that the trade-off (an unobserved-Task
  IOException/cancellation is treated as noise) is deliberate. The predicate only
  runs for events already carrying the UnobservedTaskException mechanism, so it
  never suppresses these types on a normal observed path.

No behavior change; the type set is unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@hatton

hatton commented Jul 19, 2026

Copy link
Copy Markdown
Member Author

[Claude Fable 5] Consulted Devin on 2026-07-18 19:22 up to commit c81bb65 (re-review after the review-follow-up commit).

Result: 0 bugs, 1 Investigate, 1 Informational — the same AggregateException-unwrapping Investigate flag as the prior review, which is already posted as an inline thread, refuted with a real captured event, and resolved; skipped re-posting per dedup. Greptile re-review passed with no new findings; both earlier Greptile P2 items are addressed and resolved. CI green. Re-review clean — bots quiet.

@hatton
hatton marked this pull request as ready for review July 19, 2026 01:23
JohnThomson and others added 2 commits July 23, 2026 15:50
…L-16573)

The unobserved-Task filter previously dropped an event if ANY exception in
the chain was a benign socket-abort type, which could suppress a real bug
aggregated alongside socket noise. Now at least one benign abort must be
present AND every exception in the chain must be benign (or the
AggregateException wrapper). Also documents why the mechanism check must
remain Any. Adds tests for the mixed-chain and bare-wrapper cases.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@JohnThomson
JohnThomson marked this pull request as draft July 23, 2026 20:51
@JohnThomson

Copy link
Copy Markdown
Contributor

[Claude Fable 5] Consulted Devin on 2026-07-23 up to commit d39376a (re-review after tightening the benign-type check from Any to All).

Result: 0 bugs, 1 Investigate, 2 Informational. The Investigate flag is the same AggregateException-unwrapping question raised on the previous commit — its thread already has the answer (verified against a real captured event on BLOOM-DESKTOP-EQ4: the wrapper and its mechanism survive to BeforeSend in the SDK version we ship) and is resolved. The two Informational notes restate trade-offs already documented in code comments (the breadth of the benign type set, and BeforeSend applying to the whole pipeline but being scoped by the mechanism check). No new findings to mirror.

@JohnThomson JohnThomson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JohnThomson+AGNT resolved 1 discussion.
Reviewable status: 0 of 2 files reviewed, all discussions resolved (waiting on hatton).

@JohnThomson
JohnThomson marked this pull request as ready for review July 24, 2026 13:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants