[voipmonitor]

Problem

At C32 on DeepSeek V4.1 TP4 the serving worker stalled for 10–35 ms inside RequestScheduler::plan every few seconds, and for ~0.7 s after 32 requests arrived. The worker does not poll completions while it plans, so a finished round's successor, or a probe waiting for the queue to drain, started late and the GPU idled.

perf put the time in funded acceptance learning: acceptance_probe compared each probe candidate with the ordinary winner slot by slot, and every comparison of a packed round re-estimated the whole round's prefix lengths and lane values, re-evaluating each lane's acceptance curve for every extra row. The rest of planning also repeated per-lane and per-shape work, so its cost grew roughly with the square of the request count.

Change

The same values are computed fewer times; selection and its arithmetic are unchanged.

• Acceptance probes estimate each candidate's round widths once; prefix allocation computes each lane's probabilities once.
• A request's pending advance is tracked where commit and completion already maintain its pending count, instead of scanning all submitted lanes on every use. Debug builds check it against the scan.
• Packed rounds share one ranking per draft and width bound, evaluate lane curves once per call and priorities once per shape, and sort only the leading cohort.
• Padding pruning compares only candidates with the same kind, cohort, and prefill row, estimating each candidate's lengths once.
• Speculative durations read the packed round table directly; it already includes the full B×W graph.

Equivalence

A local differential harness (not in this PR) records every plan, lane value, pruned candidate list, probe choice, and round length over seeded random scenarios: 1–64 requests, host/device/mixed prefix allocation, fixed widths, mixed graphs, grammar and host-stop lanes, queued rounds, retirement, refresh and acceptance probes. Across 1056 scenarios and ~210 000 plans the traces of main and this branch are byte-identical (one seed set also in a debug build).

Results

Median plan() time with one round queued (release test build, frank1):

Requests	Allocation	main	this PR	with acceptance probe: main	this PR
8	device	140 µs	60 µs	2563 µs	111 µs
16	device	289 µs	96 µs	2153 µs	201 µs
32	device	669 µs	196 µs	5312 µs	413 µs
64	device	1773 µs	436 µs	13380 µs	943 µs
32	host	754 µs	250 µs	1363 µs	497 µs
64	host	1957 µs	615 µs	3546 µs	1215 µs

Service at C32 on switched GPUs 0–3 of frank1 and frank2, #182 applied in both builds, two services × five 30 s cells per row, mirrored order:

Host / package	before tok/s (worst cell)	after tok/s (worst cell)
frank1 main	2471 (2428)	2524 (2490)
frank1 B	2474 (2432)	2529 (2495)
frank2 main	2478 (2398)	2537 (2503)
frank2 B	2450 (2313)	2540 (2515)
all 40 cells	2468	2532 (+2.6 %)

In a diagnostic run on frank1 the longest plan call fell from 34.4 ms to under 1 ms, no worker iteration exceeded 10 ms, and the GPU busy share of the steady window rose from 98.4–98.9 % to 99.8–99.9 %.

Validation

• Helios library tests pass in release and debug builds except five template, package-cache, and image-hint tests that fail identically on main (missing local assets).
• A new test covers the tracked pending ceiling across queued rounds, completion, and retirement.

Limitations

• While a winning probe waits for the queue to drain, the worker still plans again on every iteration; each plan is now short.

🤖 Generated with Claude Code