Split submit_request from handle_response; add async submit + get_response#648
Merged
Conversation
submit_request now does only the POST and returns the initial ResponseModel (with self.job_id recorded). Dispatching the response to handle_response is left to the caller so that async / streaming consumers can yield the initial response before any side effects (display updates, RemoteException raises) run. Add async_submit_request and async_get_response using httpx.AsyncClient so callers inside an event loop don't need a threadpool or a duplicated POST path. The new parsing helper _parse_submit_response is shared between the sync and async submits. The three internal callers (blocking_request, async_request, non_blocking_request) now invoke handle_response explicitly on the initial response to preserve today's behaviour.
JadenFiotto-Kaufman
added a commit
to ndif-team/workbench
that referenced
this pull request
Apr 20, 2026
The duplicated _async_submit in StreamingRemoteBackend existed only because the parent RemoteBackend didn't expose an async submit. With ndif-team/nnsight#648 merged, we can delete it and call super().async_submit_request() directly — which also means the job_id bookkeeping (self.job_id = response.id) is handled by the parent, not open-coded here. Depends on ndif-team/nnsight#648.
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactor `RemoteBackend` so a subclass (or an async caller that wants streaming control) can submit a request without paying for `handle_response`'s side effects, and can use `httpx.AsyncClient` instead of the sync path.
Motivated by the workbench SSE endpoint flow: the FastAPI route wants to `async yield` the initial `ResponseModel` to the browser before anything touches the terminal-display logic or raises on error, and it wants to run the POST inside the event loop without a threadpool.
Changes
Test plan
Companion
Workbench side: ndif-team/workbench#112 — its `StreamingRemoteBackend` now calls `super().async_submit_request()` instead of duplicating the POST.