Repository navigation
ci(review): give the reviewer room to answer after thinking - #60
Merged
Merged
Conversation
DeepSeek's thinking models used the whole 8000-token default on reasoning and never answered on large pull requests, which failed every hypothesis chunk. The review job now allows 32000 tokens a reply and splits the diff into 20 KB chunks.
There was a problem hiding this comment.
⭐ Aster Review
Reviewed this PR and found 1 issue (1 high)
| Severity | Category | Location | Finding |
|---|---|---|---|
high |
error-handling |
.github/workflows/review.yml:100 |
Hard-coded 32000 output tokens can exceed model output limits |
⭐ Cost & usage
| Metric | Value |
|---|---|
| Total tokens | 12.6k |
| Prompt / completion | 5k / 7.5k |
| Requests | 5 |
| Estimated cost | ~$0.0053 |
| # DeepSeek's thinking models spend the reply budget on reasoning | ||
| # before they answer. At the 8000-token default a 40 KB diff chunk | ||
| # ran out with no answer, so give each reply more room and each | ||
| # chunk less diff. |
There was a problem hiding this comment.
Hard-coded 32000 output tokens can exceed model output limits — high/error-handling · 75% confident
If the AI model chosen for the review isn't allowed to return that much text at once, every review request is refused and the bot never posts its comments.
Fix: Make the token budget per-model (e.g. ASTER_MAX_TOKENS / ASTER_VERIFY_MAX_TOKENS / ASTER_HYPOTHESIS_MAX_TOKENS) and cap it to the model's known output limit, or have the code clamp to the provider limit rather than hard-coding a global value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Review check failed on every push to #59 with "The model used its whole reply length thinking and never wrote an answer". DeepSeek's thinking models spent the whole 8000-token default reply on reasoning for each 40 KB diff chunk, so all hypothesis chunks failed.
The review job now sets:
ASTER_MAX_TOKENS=32000: more room to answer after thinking. DeepSeek's thinking models allow up to 64K output tokens.ASTER_HYPOTHESIS_CHUNK_BYTES=20000: smaller chunks, so each one needs less thinking.This PR runs the updated workflow on itself.
Summary by CodeRabbit