[PR #13561/148205f2 backport][3.15] Add masked WebSocket read benchmarks and fix read benchmark backpressure - #13564
Conversation
…ure (aio-libs#13561) (cherry picked from commit 148205f)
Confidence Score: 4/5The PR appears safe to merge, with one non-blocking benchmark-isolation concern. The benchmark frames fit comfortably below the queue threshold and are encoded correctly, but measured invocations now include reader, queue, and protocol allocation in addition to parser work. Files Needing Attention: tests/test_benchmarks_http_websocket.py Reviews (1): Last reviewed commit: "Add masked WebSocket read benchmarks and..." | Re-trigger Greptile |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 3.15 #13564 +/- ##
==========================================
- Coverage 96.27% 96.27% -0.01%
==========================================
Files 161 161
Lines 51870 51896 +26
Branches 2848 2851 +3
==========================================
+ Hits 49938 49963 +25
Misses 1752 1752
- Partials 180 181 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. |
Merging this PR will degrade performance by 94.93%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| 🆕 | test_read_one_hundred_masked_large_binary_websocket_messages[pyloop] |
N/A | 60.1 ms | N/A |
| 🆕 | test_read_one_hundred_masked_websocket_text_messages[pyloop] |
N/A | 681.3 µs | N/A |
| 👁 | test_read_one_hundred_websocket_text_messages[pyloop] |
475.3 µs | 595.7 µs | -20.21% |
| 👁 | test_read_large_binary_websocket_messages[pyloop] |
100.5 µs | 31,131.3 µs | -99.68% |
Comparing bdraco:patchback/backports/3.15/148205f2fdc477ad3e4de0439d817b2a495ccf1e/pr-13561 (c326ee8) with 3.15 (3aed27f)
Footnotes
-
86 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
This is a backport of PR #13561 as merged into master (148205f).
What do these changes do?
The read benchmarks only fed unmasked frames, so the masked path a server takes for every frame from a client was invisible to CodSpeed; this adds masked variants for small text and large binary messages. It also fixes the existing read benchmarks; they stored an AssertionError from pause_reading after the eighth large frame and measured the early return path for the remaining feeds. The parser also stops parsing once the queue crosses its limit, so each benchmark round now uses a fresh reader with a queue limit the batch cannot reach.
CodSpeed will show a large regression on the large binary read benchmark; that is really restoring the benchmark to its original design. We missed
_upgraded = Truein the original, and after #11966 went in it would latch the AssertionError from pause_reading failing, so what looked like an improvement was the benchmark doing effectively nothing after 8 frames.Are there changes in behavior for the user?
No, benchmarks only.
Is it a substantial burden for the maintainers to support this?
No, the new benchmarks follow the shape of the existing ones.
Related issue number
Noticed while benchmarking #13559.
Checklist
CONTRIBUTORS.txtN/A, already listedCHANGES/foldername it
<issue_or_pr_num>.<type>.rst(e.g.588.bugfix.rst)if you don't have an issue number, change it to the pull request
number after creating the PR
.bugfix: A bug fix for something the maintainers deemed animproper undesired behavior that got corrected to match
pre-agreed expectations.
.feature: A new behavior, public APIs. That sort of stuff..deprecation: A declaration of future API removals and breakingchanges in behavior.
.breaking: When something public is removed in a breaking way.Could be deprecated in an earlier release.
.doc: Notable updates to the documentation structure or buildprocess.
.packaging: Notes for downstreams about unobvious side effectsand tooling. Changes in the test invocation considerations and
runtime assumptions.
.contrib: Stuff that affects the contributor experience. e.g.Running tests, building the docs, setting up the development
environment.
.misc: Changes that are hard to assign to any of the abovecategories.
Make sure to use full sentences with correct case and punctuation,
for example:
Use the past tense or the present tense a non-imperative mood,
referring to what's changed compared to the last released version
of this project.
Drafted with Claude Code (Fable 5); reviewed by @bdraco.