`aioresponses` is incompatible with aiohttp 3.14.0 - switch to `aiointercept`

[adubovik]

Describe the bug

The folowing fix #12452
introduced a required field called stream_writer in the __init__ method of the public class aiohttp.client_reqrep.ClientResponse:

https://github.com/aio-libs/aiohttp/blob/master/aiohttp/client_reqrep.py#L219-L234

However, this breaking change has led to a bump of the minor package version: 3.13.5 => 3.14.0

This results in a lot of CI breaks accross many repos where the aiohttp bump happend automatically by dependabot to close security issue.

Is it possible to make a patch release 3.14.1 where stream_writer is an optional argument to preserve backward compatbility?

To Reproduce

Install aiohttp==3.14.0 and aioresponses==0.7.8, write a simple mock test, observe the failure:

    
        # We need to initialize headers manually
        _headers = CIMultiDict({hdrs.CONTENT_TYPE: content_type})
        if headers:
            _headers.update(headers)
        raw_headers = self._build_raw_headers(_headers)
>       resp = response_class(method, url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       TypeError: ClientResponse.__init__() missing 1 required keyword-only argument: 'stream_writer'

.nox/test/lib/python3.11/site-packages/aioresponses/core.py:172: TypeError

pnuckowski/aioresponses#289

Expected behavior

Bump from aiohttp==3.13.5 to aiohttp==3.14.0 must not break dependant project/packages.

Logs/tracebacks

# We need to initialize headers manually
        _headers = CIMultiDict({hdrs.CONTENT_TYPE: content_type})
        if headers:
            _headers.update(headers)
        raw_headers = self._build_raw_headers(_headers)
>       resp = response_class(method, url, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
E       TypeError: ClientResponse.__init__() missing 1 required keyword-only argument: 'stream_writer'

.nox/test/lib/python3.11/site-packages/aioresponses/core.py:172: TypeError

Python Version

$ python --version
Python 3.11.13

aiohttp Version

$ python -m pip show aiohttp
Name: aiohttp
Version: 3.14.0
Summary: Async http client/server framework (asyncio)
Home-page: https://github.com/aio-libs/aiohttp
Author: 
Author-email: 
License: Apache-2.0 AND MIT
Location: /Users/anton_dubovik/Code/GitHub/ai-dial-adapter-dial/.venv/lib/python3.11/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, propcache, typing_extensions, yarl
Required-by: aioresponses

multidict Version

$ python -m pip show multidict
Name: multidict
Version: 6.0.4
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /Users/anton_dubovik/Code/GitHub/ai-dial-adapter-dial/.venv/lib/python3.11/site-packages
Requires: 
Required-by: aiohttp, yarl

propcache Version

$ python -m pip show propcache
Name: propcache
Version: 0.2.0
Summary: Accelerated property cache
Home-page: https://github.com/aio-libs/propcache
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /Users/anton_dubovik/Code/GitHub/ai-dial-adapter-dial/.venv/lib/python3.11/site-packages
Requires: 
Required-by: aiohttp, yarl

yarl Version

$ python -m pip show yarl
Name: yarl
Version: 1.17.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /Users/anton_dubovik/Code/GitHub/ai-dial-adapter-dial/.venv/lib/python3.11/site-packages
Requires: idna, multidict, propcache
Required-by: aiohttp

OS

macOS

Related component

Client

Additional context

No response

Code of Conduct

• I agree to follow the aio-libs Code of Conduct

[webknjaz]

While we do have a SemVer strategy announced in the docs, the public API convention applies to what's documented.

Specifically, https://docs.aiohttp.org/en/stable/client_reference.html#aiohttp.ClientResponse is clear that its shape is public, but it's never expected to be constructed by the end users:

User never creates the instance of ClientResponse class but gets it from API calls.

---

That said, the SemVer promise hasn't been violated.

Additionally, the end users are the ones in control of their envs and are expected to test their env deptrees: https://hynek.me/articles/semver-will-not-save-you/#taking-responsibility.

So yes, this might be an inconvenience, but it's not a breaking change in the scope of aiohttp.

[webknjaz]

cc @bdraco I suppose this is what you mentioned earlier?

[Dreamsorcerer]

Try https://github.com/Polandia94/aiointercept instead. It's designed to replace aioresponses and doesn't rely on monkey-patching our internals. In theory, it shouldn't break with each release (I see the CI was updated to test against 3.14 with no code changes).

[hikemalliday]

Yep I have the same issue as OP with aioresponses. Bummer

[adubovik]

@webknjaz a few thoughts:

1. I believe the source code alone should communicate API boundaries. If ClientResponse isn't meant to be instantiated by users, the natural signal is a leading underscore. A public class with a public constructor is an implicit invitation. Expecting integrators to also read the docs to find out which public symbols are actually off-limits is a pretty unusual contract.

2. The docs caveat doesn't change the real-world impact. aioresponses is one of the most widely used aiohttp testing libraries. Breaking it on a minor bump is exactly what semver is supposed to prevent, footnote or not.

3. On the blog post — sure, consumers should test their dependency trees. But if aiohttp ships under semver, that comes with an obligation too: minor bumps should be safe to take. That's literally what Dependabot assumes when it opens these PRs. You can't lean on semver's reputation and then point people at "semver won't save you" when a minor release breaks them.

4. Finally, the fix is trivial. stream_writer didn't exist before 3.14 — existing callers had no way to pass it. Making it optional would restore compat at essentially zero cost.

[bdraco]

TLDR: Try migrating to aiointercept

ClientResponse is exported, so the class is public, but its constructor is not. The docs are explicit that "User never creates the instance of ClientResponse class but gets it from API calls." aioresponses breaks because it calls __init__ directly, which the docs say not to do. By replicating our constructor signature, it ties itself to an internal that we change as needed, so it breaks on nearly every minor bump.

I've sent fixes upstream before (e.g. pnuckowski/aioresponses@e909123), but I'm not going to keep doing that, for a few reasons:

• Each fix is temporary. It patches the current signature break but does nothing about the design problem, which is mocking an internal we told people not to touch, so the next signature change breaks it again and we're back here.
• It papers over the real issue and keeps users on a barely-maintained library, which is what actually causes them pain. Fixing it upstream signals "this is fine to keep using", which isn't the message we want to send.
• It's not our library to maintain. Spending maintainer time chasing its breakage on every release isn't sustainable, and adding a permanent None default to each new internal argument just to keep it working only defers the next break.

The sustainable answer is to migrate. For mocking that doesn't depend on our internals, aiointercept is a better design and has been surviving our releases without code changes. There's an open discussion with more context: https://github.com/orgs/aio-libs/discussions/45

[Dreamsorcerer]

2. Breaking it on a minor bump is exactly what semver is supposed to prevent, footnote or not.

Breaking a library that monkey patches our internal code and ignores all our best practices is absolutely not what semver is supposed to prevent...

4. Finally, the fix is trivial. stream_writer didn't exist before 3.14 — existing callers had no way to pass it. Making it optional would restore compat at essentially zero cost.

This would likely break the code in aiohttp 3.14, or at best leave the possibility of future breakages where the stream_writer was mistakenly not passed when it was actually required.

Again, try aiointercept. My understanding is that it should be a couple of lines for an initial migration. If it works out well, and users are not having problems with it, then we'll likely bring that library in as a first-party library and maintain it ourselves. But, we can't maintain aioresponses.

[webknjaz]

1. I believe the source code **alone** should communicate API boundaries. If `ClientResponse` isn't meant to be instantiated by users, the natural signal is a leading underscore. A public class with a public constructor is an implicit invitation. Expecting integrators to also read the docs to find out which public symbols are actually off-limits is a pretty unusual contract.

Ecosystem conventions aren't enforceable contracts. If the class was underscored, people would find other ways to complain, like linters being unhappy when they use it in type annotations. If you're okay with relying on implicit assumptions, though, feel free to rely on the assumption that you shouldn't misuse the internals 🤷‍♂️

Python doesn't have a standard way of marking what's public, short of __all__ and a single leading underscore. But again, these conventions are something someone would follow, while others wouldn't. And more often than not people still monkey-patch "_private" interfaces because it works for them (until it doesn't). Hyrum's Law is a thing but what's public and what isn't is still up to the maintainers, regardless of whether somebody's implicit assumption of where the public interface line is drawn.

We should probably just go ahead and follow pip's and pytest's structure putting everything under a private namespace but right now, it is what it is due to historic reasons.

2. The docs caveat doesn't change the real-world impact. `aioresponses` is one of the most widely used aiohttp testing libraries. Breaking it on a minor bump is exactly what semver is supposed to prevent, footnote or not.

SemVer doesn't actually work universally. See the blog post again. Parties will never have exactly the same assumptions of what's public because the perception is subjective and there are other widely accepted legitimate reasons for making unexpected changes like making security fixes.

3. On the blog post — sure, consumers should test their dependency trees. But if aiohttp ships under semver, that comes with an obligation too: minor bumps should be safe to take. That's literally what Dependabot assumes when it opens these PRs. You can't lean on semver's reputation and then point people at "semver won't save you" when a minor release breaks them.

SemVer's reputation is what it doesn't work and people misunderstand its promises. Despite that, it's being followed and people still have their own ideas of what other people should do every time they have to change their project because another project changed regardless of whether a bump was major or not. In fact, I'd argue that it should be expected any release with security updates will inevitably break somebody.

4. Finally, the fix is trivial. `stream_writer` didn't exist before 3.14 — existing callers had no way to pass it. Making it optional would restore compat at essentially zero cost.

I guess if you can convince @Dreamsorcerer that it's fine, we might entertain such a pull request provided that it includes full code coverage and a deprecation warning, which would probably require handling a missing arg in all the code paths that rely on it. But again, this is trying to accommodate for a third party thing that abuses monkey-patching a private API (while also implementing a fragile-by-design testing practice).