Skip to content

Commit ea7e971

Browse files
committed
Consult ignore_eos when classifying a finish reason at the output limit
A request with ignore_eos suppresses end-of-sequence termination, so its last sampled token can be an EOS id while it finishes on the output limit. The terminal event classified that as a stop with Eos termination, which tells clients a truncated generation completed normally. Use the request's ignore_eos setting in the finish-reason classification, the same way the emitting and logprob paths already do, and document the finish reason the client sees for each termination cause. Validation: release Helios library suite on the build host, 1118 passed and 5 failed; the same tree without this change, with the new regression case applied, is 1117 passed and 6 failed, and the five shared failures are the pre-existing engram staging and package-cache environment failures reproduced on the unmodified tree. Both builds also served one prepared single-GPU package side by side: with greedy sampling, a request that reaches its output limit on an EOS token reports stop on the unmodified build and length on this one, 6/6 each way, while requests that honour EOS, stop before it, or ignore it without ending on it report the same reason on both builds. Material limits: the regression drives the simulated worker phases, and the sampled-EOS suppression itself is unchanged. Design-Signed-Off-By: Hermes Agent (DeepSeek V4.1 Flash); design-review=2d93
1 parent ccf012d commit ea7e971

3 files changed

Lines changed: 11 additions & 11 deletions

File tree

‎docs/TODO.txt‎

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -98,9 +98,6 @@
9898
[ ] Qualify prompt scoring on remaining recipes and cache lifecycles;
9999
see docs/logprobs-design.md for TP4 costs and numerical limits.
100100

101-
[ ] Respect ignore_eos when classifying EOS at the output limit;
102-
see helios/src/server/runtime.rs.
103-
104101
[ ] Validate embedding readiness timestamps in full Qwen3.8 PLE serving and
105102
cross-node RoCEnante TP packages for Qwen3.8 and DeepSeek V4.1.
106103

‎docs/api.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -352,6 +352,13 @@ or partial tags. Structured answers become verbatim when ordinary answer content
352352
begins, preserving scalar JSON, Lark output, and markup inside string values.
353353
Shared runtime handling owns token accounting, stop filtering, and termination.
354354

355+
Termination supplies the client finish reason: `length` at the output limit,
356+
`tool_calls` when parsed calls complete an answer that the user did not stop, and
357+
`stop` for a matched token or string stop or a terminating end-of-sequence token.
358+
A request admitted with `ignore_eos` does not treat end-of-sequence tokens as
359+
terminating, so one that exhausts its output limit reports `length` even when its
360+
last generated token is end-of-sequence.
361+
355362
A reasoning delimiter that the tokenizer encodes as one token, such as
356363
`<think>`, `</think>`, `</mm:think>`, or Gemma 4's `<channel|>`, opens or
357364
closes reasoning only when the model generates that token. When a model writes

‎helios/src/server/runtime.rs‎

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1594,15 +1594,11 @@ impl<'a> Worker<'a> {
15941594
);
15951595
}
15961596
}
1597-
let reason = if user_stop
1598-
|| request
1597+
let eos = !request.incoming.prompt.ignore_eos
1598+
&& request
15991599
.last_token
1600-
.is_some_and(|token| self.config.eos_token_ids.contains(&token))
1601-
{
1602-
"stop"
1603-
} else {
1604-
"length"
1605-
};
1600+
.is_some_and(|token| self.config.eos_token_ids.contains(&token));
1601+
let reason = if user_stop || eos { "stop" } else { "length" };
16061602
if request
16071603
.incoming
16081604
.events

0 commit comments

Comments
 (0)