Skip to content

ci(review): give the reviewer room to answer after thinking - #60

Merged
Zfinix merged 1 commit into
mainfrom
ci/review-output-budget
Oct 25, 2026
Merged

Zfinix merged 1 commit into
mainfrom
ci/review-output-budget

Conversation

@Zfinix

@Zfinix Zfinix commented Oct 25, 2026 •

Copy link
Copy Markdown
Owner

The Review check failed on every push to #59 with "The model used its whole reply length thinking and never wrote an answer". DeepSeek's thinking models spent the whole 8000-token default reply on reasoning for each 40 KB diff chunk, so all hypothesis chunks failed.

The review job now sets:

  • ASTER_MAX_TOKENS=32000: more room to answer after thinking. DeepSeek's thinking models allow up to 64K output tokens.
  • ASTER_HYPOTHESIS_CHUNK_BYTES=20000: smaller chunks, so each one needs less thinking.

This PR runs the updated workflow on itself.

Summary by CodeRabbit

  • Chores
    • Updated automated review settings to allow larger token limits and diff chunks. These changes affect the review workflow; no end-user-facing application changes are included.

DeepSeek's thinking models used the whole 8000-token default on reasoning and never answered on large pull requests, which failed every hypothesis chunk. The review job now allows 32000 tokens a reply and splits the diff into 20 KB chunks.
@coderabbitai

coderabbitai Bot commented Oct 25, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 819309cb-0d07-49bf-86d2-cd182141c5b8
Commits

Reviewing files that changed from the base of the PR and between 8c9eabc and 1bfd76d.

Files selected for processing: 1
  • .github/workflows/review.yml

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

The Review and comment step now sets a 32,000-token reply limit and a 20,000-byte hypothesis diff-chunk limit. Comments relate these limits to DeepSeek thinking models.

Changes: Review workflow limits

Layer / File(s) Summary
Set review limits
.github/workflows/review.yml
The Review and comment step sets ASTER_MAX_TOKENS to 32000 and ASTER_HYPOTHESIS_CHUNK_BYTES to 20000. Comments describe the limits in relation to DeepSeek thinking models.

Priority: Normal

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Bug fix

Merge Risk: Minimal · up to 1bfd7

The workflow limits are configured, and no confirmed merge-blocking failure remains. Provider acceptance of the larger reply limit has not been demonstrated.

Pre-merge checks: Passed 5
Check name Status Explanation
Title check Passed The title clearly and concisely describes the main change: increasing review capacity after model reasoning.
Description check Passed The description explains the failure, identifies the cause, and documents both workflow changes. It omits the Related issues and Checklist sections, but the main change and rationale are complete.
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

Finishing Touches
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@aster-review aster-review Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⭐ Aster Review

Reviewed this PR and found 1 issue (1 high)

Severity Category Location Finding
high error-handling .github/workflows/review.yml:100 Hard-coded 32000 output tokens can exceed model output limits
⭐ Cost & usage
Metric Value
Total tokens 12.6k
Prompt / completion 5k / 7.5k
Requests 5
Estimated cost ~$0.0053

# DeepSeek's thinking models spend the reply budget on reasoning
# before they answer. At the 8000-token default a 40 KB diff chunk
# ran out with no answer, so give each reply more room and each
# chunk less diff.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hard-coded 32000 output tokens can exceed model output limits — high/error-handling · 75% confident

If the AI model chosen for the review isn't allowed to return that much text at once, every review request is refused and the bot never posts its comments.

Fix: Make the token budget per-model (e.g. ASTER_MAX_TOKENS / ASTER_VERIFY_MAX_TOKENS / ASTER_HYPOTHESIS_MAX_TOKENS) and cap it to the model's known output limit, or have the code clamp to the provider limit rather than hard-coding a global value.

@Zfinix
Zfinix merged commit 9455592 into main Oct 25, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant