[brandonmmusic-max]

brandonmusic/glm-5.3-flash-tr3-fp8 is a TrellisMX release of GLM-5.3 Flash, quantized directly from
zai-org/GLM-5.3-Flash-BF16 with the procedural NPK codec. Flare could not resolve, plan, verify or
prepare it: the GLM recipe reads only the NVFP4 checkpoint. This change serves it at the maintainer's
request. The release is a second checkpoint of the GLM-5.3 Flash recipe, glm-5.3-flash-trellismx;
ignition export glm-5.3-flash-trellismx prepares it and flare run serves it on four GPUs. The
release, the codec and the P8 kernels are Brandon M. Music's; the b12x W4A8 trellis path they build
on is Luke Alonso's.

Change

• Checkpoint. The release overlays its routed experts on a pinned carrier: carrier/ holds every other tensor
  as stored (BF16 projections and head, FP32 scalars), and one sidecar per layer and TP4 rank holds the routed
  experts, so only TP4 can load it. Checkpoint gains tensor_parallel and overlay; local release directories
  resolve to the TrellisMX checkpoint, and cached and post-download verification require the canonical inventory
  (one sidecar per allocated layer and rank at its declared size, plus the encoder transform). The recipe pins
  release revision 80cbe535, whose carrier chat template is byte-identical to the NVFP4 checkpoint's, so the
  recipe's insertion patch annotates it and preparation checks both the source and the result digest.
• Detection and loading. Detection checks the manifest, the carrier pin and config, the quantization config
  and every sidecar header against the coupled contract. Loading reads each sidecar once, checks its size and
  SHA-256 against the manifest, and builds the tensors from those same bytes. Every rank compares manifest and
  transform digests before any of them loads.
• Execution. b12x's TrellisMX source format prepares the routed experts, and b12x's in-tree P8 runtime (the
  prepared-program runtime for these experts) executes them. Preparation builds the runtime with b12x's declared
  GLM-5.3 Flash serving configuration (GLM53_SERVING_OPTIONS, which the recipe reads from b12x rather than
  restating), reads nothing from the environment, and attaches it under b12x's in-tree backend declaration once
  the runtime has resolved that configuration. The launch geometry is not a recipe setting: each capacity's plan
  races b12x's P8 launch configurations and fixes the winner before capture, and the plan declares the arena
  every P8 call carves its buffers from, which preparation sizes in the shared scratch arena. Attaching runs no
  runtime call: the runtime declares every program that a call of 1 to its capacity in rows reaches, and b12x
  preparation compiles those programs and installs them in the runtime before it freezes kernel resolution, so
  no capture compiles or allocates. Compile workers receive b12x's inert descriptor; only the preparing process
  runs the runtime. The routed-expert record names the release that supplied its experts, and the TrellisMX
  geometry checks read that typed origin. Each rank records the backend identity, b12x's serving configuration
  and the attachment identities with their resolved settings, and refuses to report if a TrellisMX layer has no
  live runtime. The mHC prefill route is left to b12x's autotuner, like every other recipe's.
• Memory. The checkpoint declares the resident bytes preparation keeps: 10,046,638,288 bytes of expansion,
  which a release test checks against the prepared bundle, and 4,457,303,696 draft bytes.
• Qualification. scripts/qualify_glm53_trellismx.py runs the release's two test modules with the GPU and
  release cases enabled and fails unless every collected test ran and passed.
• No Protocol Buffer schema or Helios code changes; packages exported before this change load as before.

Dependencies

• local-inference-lab/flashinfer, "b12x fused MoE: TrellisMX (procedural NPK) source format with an in-tree P8
  runtime": required to prepare and serve the release. pyproject.toml and uv.lock are unchanged and keep Flare's
  FlashInfer @main pin, so the release cannot be prepared until that change is on main.
• local-inference-lab/flashinfer, "b12x mHC: keep FP32 mixing weights in the TF32 projection": keeps the mixing
  coefficients at FP32 accuracy when b12x's autotuner picks the TF32 mHC route; all figures in Validation used it.
• Flare, "Drop reasoning close delimiter tokens from visible content": GLM-5.3 Flash can generate its </think>
  token after reasoning has ended, and without that change the token appears in message.content.

Validation

• CPU (CUDA_VISIBLE_DEVICES="", companion FlashInfer changes c37d2d9ee and 9be301a77 installed): no test fails
  only with this change.
• GPU, one RTX PRO 6000 Blackwell Max-Q: scripts/qualify_glm53_trellismx.py collected 192 tests and passed all 192,
  none skipped, in 26 minutes.
• TP4 end to end on four RTX PRO 6000 Blackwell GPUs (two Max-Q, two Workstation Edition), no mHC route pin, with
  this change applied to NVIDIA main a161b17f: the export from an empty kernel cache passed in 41 minutes and the
  package was signed; 443 of 469 reliability requests at the model card's sampling answered correctly, and none of
  the 373 whose content was recorded in full carried a think tag; decode ran at 189.2 tok/s for one stream and
  751.9 tok/s for 32 streams at context 0, Sieve coding peak median 229.7 tok/s.

Evidence

• CPU: the full tests/ directory gives 64 failed, 4,326 passed and 2,515 skipped on this branch against 64 failed,
  4,180 passed and 2,466 skipped on unmodified 6fa01a49 in the same environment, with identical failing ids; the
  failures are this environment's (template fixtures and native-recipe contract generation are not set up here).

• GPU: the qualification runs against the release at 80cbe535; its release cases detect, load and verify the
  published sidecars.

• TP4 end to end: FlashInfer f5f5341d7 plus the companion changes (TrellisMX c37d2d9ee, mHC precision
  9be301a77); apart from the mHC fix, that run's FlashInfer runtime differed from c37d2d9ee in comments, one
  validation error message and a trailing-padding clear span this geometry does not have, its FlashInfer tests predate
  the tail test in c37d2d9ee, and its Flare tree held this change with a test-only difference. The build also carried
  changes this PR does not contain: the mixed Max-Q/Workstation tuning agreement (Treat Max-Q and Workstation Edition device names as one GPU when tuning #150), retained warmup mapping reuse with
  its follow-up fixes (Reuse retained warmup mappings across compiled warmup #154), the reasoning close-token output fix, and two local-only changes for this host (an
  override of torch's compile-package GPU-name comparison in ignition/preparation/aot.py for the two card names,
  and acceptance of this host's DeepSeek checkpoint's 2-D slab scale planes in ignition/components/weights.py and
  ignition/preparation/csf_checkpoint.py).

  • Export: all 1,287 prepared requests and 122 serving graphs; every rank verified its sources.
  • Boot healthy in 104 s. Smoke NLL 1.5618 over 12 of 16 scored positions, equal to the export of this release
    made before this change. The smoke's greedy chat check spent its 512-token budget inside reasoning and returned
    no content.
  • Decode tok/s (llm-inference-bench v0.4.29), context 0 / 8K: C1 189.2 / 218.4, C2 281.0 / 283.5, C4 377.2 /
    383.7, C8 471.9 / 462.8, C16 594.5 / 590.3, C32 751.9 / 697.8, no errors; speculative acceptance over the bench was
    0.68 (112,117 of 163,792 drafts, from the server's counters). Sieve coding peak median 229.7 tok/s (5 of 5).
  • Reliability: 12 bursts of 32 or 28 requests that restore one published prompt and 13 prompts of 99,208 tokens
    sent one at a time, 373 requests: 351 correct, 8 with an adjacent record number, 4 with a wrong record, and 10 the
    fixture grades corrupt (6 that spent the 1,024-token budget in reasoning, 3 that ended with the answer inside
    reasoning and no content, and 1 whose content repeats the right answer twice); 11 of the 13 long prompts were
    correct, and none of the 373 replies carried a think tag in its content. Three bursts of 32 distinct prompts: 92
    of 96 correct. No request or device errors.
  • Close-delimiter checks: 64 streamed and 64 token-id requests, no close delimiter in visible content; a tool call
    parsed.

  These figures measure the whole local system, not this change alone.

Limitations

• Validated on SM120 (RTX PRO 6000 Blackwell) only; SM103, SM121 and Windows were not run.
• The manifest lists no carrier digests, so carrier files get header checks only, as every other checkpoint does.
• The release ships TP4 sidecars only, so planning offers TP4 only.

Related pull requests

• FlashInfer TrellisMX source format and P8 runtime: b12x fused MoE: TrellisMX (procedural NPK) source format with an in-tree P8 runtime local-inference-lab/flashinfer#69
• FlashInfer mHC TF32 projection precision: b12x mHC: keep FP32 mixing weights in the TF32 projection local-inference-lab/flashinfer#68
• Reasoning close-delimiter output fix: Drop reasoning close delimiter tokens from visible content #162