[tunamitom]

• base: voipmonitor/flare release/k31-exl3-tp8-20261108 (intake lane, tip 1d4007f)
• head: tunamitom/flare-local-inference pr/flare-pr9-route-rows-9-20-lane @ 95757eeb
• title: kimi3: route 9..20 decode rows through the pair gather + standalone top-16 (numerics change on re-export)

What / why

Above eight rows the recipe left the route exchange entirely (latent
all-reduce rows x 3584 BF16, router all-gather, two copies, native route
top-k). MAX_ROWS 8 -> 20 keeps decode rounds of 9-20 rows (two
concurrent speculative streams up to 2 x (depth + 1) rows, or
four-stream rounds up to 20) on the route exchange: paired gather plus
the standalone top-16; the fused single-exchange select stays at <= 8
rows. The decoder routes every 9-20-row round onto this path
unconditionally — there is no FlashInfer pin detection and no feature
fallback: the recipe prepares 1-20 rows or raises at preparation.

Claims and caveats

• Not push-gated: the route path runs on whatever paired-gather
  transport the b12x plan selects. The per-layer speed evidence
  (30.75 / 38.35 / 48.57 / 59.63 us saved at 9 / 12 / 16 / 20 rows;
  slowest-rank medians, >= 200 replays x 5) was measured on a
  push-planned exchange. The pull transport at 9-20 rows is
  unmeasured — that is the path a default upstream deployment takes at
  these widths. The push path carries the evidence; no speed claim is
  made for the pull transport, and measuring it at 9/12/16/20 rows
  (slowest-rank medians, same method) is a stated follow-up before this
  path is relied on by default.
• Numerics change on re-export at 9-20-row rounds (flagged in the
  title): the latent arrives by gather instead of the BF16 all-reduce —
  2.63e-3 relative RMS vs the general path at every width (one BF16
  unit roundoff of the general path's partial sums); the route path is
  strictly closer to FP32 (1.66e-3 vs 2.36e-3). Ids identical at
  9/10/12/16 rows (6 banks x 8 ranks, weights within 5 fp32 ULP); one
  near-tie 16th-slot swap row in 960 at 20 rows (near-tie gate over 3.2M
  rows: every set difference a 1-for-1 swap within 2 fp32 ULP, set rate
  <= 5e-5, 0 violations). Existing captured packages are unaffected;
  re-exported packages change arithmetic at those widths — maintainer
  acceptance required.

Dependency — flashinfer PR NVIDIA#102 (standalone top-16 select)

The standalone top-16 in the FlashInfer pin must accept the full 1-20
row range (the pin declares KIMI_TOPK16_MAX_ROWS = 20). The actual
relationship to the two upstream changes: their route-gather PR (NVIDIA#101)
covers 9-16 rows; this PR extends the recipe to 20 rows and needs NVIDIA#102's
standalone select (the 17-20-row work, now open as
local-inference-lab/flashinfer#102). The pin bumps when NVIDIA#102 merges;
until then, preparation for 9-20 rows requires it — on an older pin
the top-16's row check raises at preparation of those widths, loudly and
by design. There is no feature detection and no fallback to the general
path: 9-20-row rounds either route with the pinned select or preparation
fails visibly. Merge blocker: the pin bump lands with or immediately
after NVIDIA#102 merges.

Numerics class

NUMERICS-LIGHT, BF16-structural (see above).

Tests

• tests/recipes/test_kimi31.py — 33 passed (route-gather surface
  selections at 1/2/8/9/12/16/20 rows: one row fuses the select, two to
  twenty rows pair the exchange with the standalone top-16; a 21-row
  envelope-rejection check; the 25 existing tests). With
  test_kimi3_preparation.py: 57 passed vs base's 49 (the 8 new
  items; base-failure parity holds — no test that passes at base fails
  here). Re-run on the final head in this pass.
• The Flare test mocks the b12x plan surfaces: it proves which surfaces
  are planned, not results. The GPU evidence is the flashinfer side's
  CUDA tests for the top-16 row-range change (Consult ignore_eos when classifying a finish reason at the output limit NVIDIA/hyperflare-inference#102); no Flare-level
  GPU route test was run on this tree. The earlier end-to-end round
  numbers were measured with a full settings bundle and are dropped
  rather than misattributed to this change.
• ruff: zero new findings vs base.

Dependencies

FlashInfer PR NVIDIA#102 (local-inference-lab/flashinfer#102, standalone
top-16 select extended to 20 rows) for the pin bump; their route-gather
PR NVIDIA#101 covers 9-16 rows, ours extends the recipe to 20 and needs NVIDIA#102's
standalone select. Preparing 9-20-row packages requires the NVIDIA#102 pin
until it merges (dependency note above).