feat(ai): expose inputMessages and partialText in onAbort callback#14141
Open
Hovo-Dev wants to merge 5 commits intovercel:mainfrom
Open
feat(ai): expose inputMessages and partialText in onAbort callback#14141Hovo-Dev wants to merge 5 commits intovercel:mainfrom
Hovo-Dev wants to merge 5 commits intovercel:mainfrom
Conversation
When a streamText call is aborted, the onAbort callback now receives usage (current step) and totalUsage (all steps combined) containing whatever real token data the provider had reported before the abort. If the provider had not yet sent a usage chunk, the values are undefined — no estimation is used. The OpenAI provider is updated to emit its finish chunk (which carries real usage) as soon as usage data arrives in the stream, rather than waiting for the stream to fully close. This ensures usage is available even when an abort fires before natural completion. Fixes vercel#7628 Fixes vercel#7805
The finish chunk was unreachable — OpenAI sends usage in a chunk with empty choices[], so choice?.delta == null triggered an early return before the emission block. Move the check before the delta guard.
2dfe4e0 to
de7c3a3
Compare
Author
|
Hi @gr2m, Took a look at this and managed to get it working cleanly — happy to share the approach. The SDK was already holding the token data we needed, it just wasn't being handed off when a stream was aborted. We also noticed that usage from OpenAI arrives mid-stream but was only forwarded at natural close, so we made sure it gets captured as soon as it arrives. That way, whether the stream finishes or gets cut short, the data is there. Looking forward to your feedback! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
When a
streamTextcall is aborted — viaAbortSignal, the user stopping generation, or a dropped connection — theonAbortcallback fires but receives no usage data.onFinishnever fires on abort at all. This makes it impossible to track token consumption for billing or quota purposes when streams don't complete naturally.Summary
packages/ai/src/generate-text/stream-text.tsusage?: LanguageModelUsageandtotalUsage?: LanguageModelUsagetoStreamTextOnAbortCallbackinputMessages: ModelMessage[]— the full message list passed to the model for the aborted step (initial messages + completed prior step toolcall/result pairs)
partialText?: string— the text actively being streamed at abort time;undefinedif no text was generated; resets at each step boundarycurrentStepUsage,completedStepsUsage,currentStepInputMessages, andpullLevelTextContentas outer-scope tracking variablesabort()function — if the provider had not yet sent usage data before the abort,usageisundefined(honest, noestimation)
packages/openai/src/chat/openai-chat-language-model.tsflush(), which never runs on aborttransform(), with afinishSentguard to prevent double-emission inflush()content/docs/07-reference/01-ai-sdk-core/02-stream-text.mdxUusage,totalUsage,inputMessages, andpartialTextfields in theonAbortcallback parameterscontent/docs/03-ai-sdk-core/50-error-handling.mdxonAbortsection to describe all four new fieldsUsage
Manual Verification
Verified via automated tests that exercise the full stream processing pipeline:
All 2200+ existing tests continue to pass.
Checklist
Related Issues
Fixes #7628
Fixes #7805