Skip to content

Browser Q4_0 load traps during CPU warmup despite GGUF/GPU format support #529

Description

@benjamin-small

Observed behavior

With a8cb07c, a locally fine-tuned SmolLM2-360M Q4_0 GGUF failed inside FlareEngine.load with WebAssembly.RuntimeError: unreachable, before init_gpu() could run. Q8_0 loaded successfully. This is the browser CPU warmup/prefill path, not the existence of a Q4_0 WGSL decode kernel.

Reproduction and cause

A reproducible Q4_0 model is included in our public stix-agent-v0.1.0 release; the release lock pins the archive/files. The model file SHA-256 is b6bc4d842fbac3c9edc8c96c689f73ac834c17969adf03ebf75ac19d4d31114d.

Use that model with upstream flare-web bindings built from a8cb07c (resolve the nested-rustdoc JS syntax problem if using wasm-bindgen 0.2.117), then call:

await init();
const bytes = new Uint8Array(await (await fetch(localModelUrl)).arrayBuffer());
const engine = FlareEngine.load(bytes); // traps before GPU initialization

The release also contains our patched runtime; using that runtime will not reproduce the original failure.

The browser loader skips f32 copies of layer matrices, retaining raw quantized data. Its CPU warmup/prefill cannot consume those raw Q4_0 layer weights in the tested path, whereas Q8_0 works.

Compatibility workaround

Our WASM-only expansion converts each 18-byte Q4_0 block to a 34-byte Q8_0 block: preserve the f16 scale, emit the 16 low nibbles minus eight, then the 16 high nibbles minus eight as signed bytes. Decoded values are unchanged. Apply to initial loading and separate raw-weight attachment. Tests cover nibble order/scales and malformed lengths.

This is a compatibility workaround, not native Q4 CPU support: both CPU and GPU use the expanded representation and consume more memory. Direct CPU Q4_0 kernels or an explicit unsupported-format error may be better upstream fixes. Browser loading, warmup, prefill and async generation should be tested together. Related #161/#165 add a GPU kernel; #278 tests block dequantization; neither establishes this browser load path.

Downstream implementation and validation

We run a patched Flare build in STIX Meta Explorer. The publicly committed patch set applies to a8cb07cf54fa43c4e41664316498ed195fa7b018 (flare-web 0.2.21); its SHA-256 is b50f6a694c85e2a74c8a4b9a7690ffa103db01d85eb8014a47e4a373f360b3e4. This is a downstream patch set, not a standalone Flare fork or an upstream-ready PR. It contains several fixes; the relevant section is linked above.

Validation notes and build/training instructions describe the tested scope. Measurements were recorded during September 8–9 development; this report is being filed September 21 after checking that upstream main is still at the same SHA. This report does not claim coverage of every model, quantization, or GPU.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions