Skip to content

fix(model): recognize F8_E4M3 GGUF tensors from ggml v0.29.0 - #9795

Open
NotAFlightRisk wants to merge 1 commit into
morluto:mainfrom
NotAFlightRisk:fix/gguf-f8-e4m3
Open

NotAFlightRisk wants to merge 1 commit into
morluto:mainfrom
NotAFlightRisk:fix/gguf-f8-e4m3

Conversation

@NotAFlightRisk

@NotAFlightRisk NotAFlightRisk commented Nov 10, 2026 •

Copy link
Copy Markdown

Summary

inspect-model-artifact doesn't know GGUF tensor type 43 yet, so an FP8 model's weights come back as UNKNOWN_43 with no byte length, and the tensor ranges and storage sizes drop to partial. ggml v0.29.0 added GGML_TYPE_F8_E4M3 = 43 on 4 November (llama.cpp v0.9.0 shipped it the same day), and llama.cpp's converter now keeps FP8 checkpoint weights in that type by default. This adds the missing row to the type table.

No linked issue - it's a one-row table update.

Problem and expected behavior

Any GGUF with an F8_E4M3 tensor triggers it, e.g. one written by convert_hf_to_gguf.py from an FP8 checkpoint (its default --outtype auto keeps FP8 weights as they are).

The table follows the ggml enum, which now has GGML_TYPE_F8_E4M3 = 43, with a block size of 1 and a one-byte ggml_fp8_e4m3_t. gguf-py in llama.cpp v0.9.0 lists it as (1, 1) as well. Expected: those tensors are named F8_E4M3 and get a derived byte length, the same as F16 or BF16.

Change and scope

Adds 43: ["F8_E4M3", 1, 1] and moves the table's pin comment from ggml da8e2a54 to 57e2c2ca (the v0.29.0 tag). Between those two commits the ggml_type enum only gains this one line, and every QK* define and block sizeof assertion in ggml-common.h is unchanged, so nothing else in the table moves. The Q8_1 note and the links in docs/model-artifacts.md stay on da8e2a54, since the layouts they cite haven't changed.

Scope decision: not applicable, it's an ordinary fix inside the existing GGUF inspection.

Contract and boundary impact

  • Semantic owner and earliest changed stage: src/model/ModelTensorTypes.ts, read by GgufHeader.ts. Nothing else interprets tensor type IDs.
  • Contribution scope fit and linked maintainer decision: ordinary fix, not applicable.
  • CLI and MCP/tool-catalog contract: no schema change. type is already a free string and type_id a number.
  • Provider, bridge, target-format, or platform compatibility: GGUF only. Every other type ID reads the same as before.
  • Evidence, artifact, provenance, or reconstruction contract: type 43 tensors now get a derived byte_length and storage_validation: "consistent". Tensor data is still not interpreted.
  • Target selection, authority, upstream behavior, and cleanup details (if applicable): not applicable.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Expected effects (check known effects; use documentation-only on its own).
Unchecked items do not establish absence; leave uncertainties visible.

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: one F8_E4M3 row in the existing "retains established storage for stable GGUF types" test in tests/boundary/model/modelArtifact.test.ts. It's 3 bytes at offset 64, so the expected total goes from 30 to 33.

  • Base reproduction or other evidence: with main's table the new row fails with type: "UNKNOWN_43", byte_length: null and storage_validation: "unknown". With this change it passes.

  • User-visible CLI/MCP output (if applicable): rea inspect-model-artifact fp8-demo.gguf --format json, on a GGUF with one F32 tensor and one 32-element F8_E4M3 tensor.

    Before (excerpt):

    {"name": "blk.0.attn_q.weight", "type": "UNKNOWN_43", "type_id": 43, "byte_length": null, "byte_length_state": "unknown", "storage_validation": "unknown"}
    "totals": {"tensor_bytes": null, ...}
    "coverage": {"tensor_ranges": "partial", "storage_sizes": "partial", ...}
    limitation: "Unknown tensor type UNKNOWN_43 prevents storage-size validation."
    

    After (excerpt):

    {"name": "blk.0.attn_q.weight", "type": "F8_E4M3", "type_id": 43, "byte_length": 32, "byte_length_state": "derived", "storage_validation": "consistent"}
    "totals": {"tensor_bytes": 48, ...}
    "coverage": {"tensor_ranges": "complete", "storage_sizes": "complete", ...}
    
  • Remaining proof gaps: I haven't run it on a full converted FP8 model. The CLI file above is a 256-byte GGUF v3 I wrote to the spec, not one written by llama.cpp.

Boundary regressions: format assumptions, with the producer revision (ggml v0.29.0) named in the table comment. The test runs ModelArtifactProvider on a written GGUF file, and the CLI run above covers the public path. Lifecycle regressions: not applicable.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

The byte length is still marked derived, and type IDs rea doesn't know are still reported as unknown.

Validation performed

Linux, Node.js 24.22.1, base 55b6d2c.

  • npx vitest run tests/boundary/model/modelArtifact.test.ts -t "retains established storage": 1 failed with main's ModelTensorTypes.ts, 1 passed with this change.
  • npx vitest run on the three tests/boundary/model/ files: 145 passed.
  • npm run check:fast: passed.
  • npm run format:check: passed.
  • node scripts/verify/architecture-guards.mjs: passed.
  • npm run lint:typed: nothing reported on the two changed files.
  • npm run build:cached, then the rea inspect-model-artifact run above, on main and on this branch.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. Type 43 tensors that were UNKNOWN_43 are now F8_E4M3.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable, Linux only.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: read-only header parsing, no new effects.

Author readiness

Human owner: @NotAFlightRisk

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-10T08:25:43.084143Z 0a543ce PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant