feat(rocm): add a GPU vendor concept and AMD device reporting - #1883
Merged
Merged
Conversation
This was referenced Sep 14, 2026
An AMD GPU resolved to `silicon_gen: Unknown` with every other field at its default, so nothing could report the device, and `cuda_arch_startup_summary` printed `sm_115` for it because MLX's ROCm backend fills `compute_capability_major`/`minor` from the HIP device properties. Vendor is a new `GpuVendor` field rather than an `AppleSiliconGen` variant. Twelve call sites read `silicon_gen != Unknown` as "not Apple Silicon" to gate a bf16 to f16 weight conversion, so a new variant there would have enabled that conversion on a backend where bf16 is native. They now call `is_apple_silicon()`, whose body is the same comparison, and a test pins that no non-Apple vendor makes it true. The vendor comes from the resolved MLX backend, exposed as `gpu_backend_kind()`, not from which `device_info()` keys are present: every backend publishes `architecture` in its own vocabulary (`gfx1151`, `sm_89`, an Apple GPU family string), so key presence alone says "AMD is CUDA". `cuda_compute_capability` is gated on that backend, and the two tests whose premise was "Metal, CUDA, or CPU-only" now ask the backend instead. New `rocm_arch` module, deliberately not a reuse of `cuda_arch`: CUDA coverage is an ordering with a PTX JIT fallback, HIP coverage is set membership with none. A list containing a generic target (`gfx11-generic`) makes the check report nothing rather than refuse, since those do load on several specific targets. `resolve_available_memory` keeps its order and its laziness, both now pinned by tests. `unified_memory_gb` stays Apple-only so the CUDA and ROCm estimate cannot move; device memory is a separate field. `hardware_label()` is unchanged: it collapses to `Unknown-0c` on both CUDA and ROCm, but it is persisted in the MTP hint files and any new spelling discards every profile recorded under the old one, so the vendor is reported alongside it in the `/v1/internal/mtp-policy` body instead. Validated on the gfx1151 spike host; the two tests this issue names are run directly because the gate run still aborts on the nvfp4 case in #1806. The cross-architecture refusal is covered by unit tests, not by a binary built for another `gfx` target. Closes #1805. Refs #1801, #1803, #1806.
inureyes
force-pushed
the
feature/issue-1805-gpu-vendor
branch
from
September 15, 2026 00:40
6c66508 to
c683b91
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1805. Part of #1801, phase 1.
What was wrong
An AMD GPU resolved to
silicon_gen: Unknownwith every other field at its default, so the startup diagnostics, the device listing and the server metadata had nothing to report. Worse,cuda_arch_startup_summaryprintedCUDA compute capability 11.5 (sm_115)for agfx1151card, because MLX's ROCm backend fillscompute_capability_major/minorfrom the HIP device properties andcuda_compute_capabilityread them as if they were CUDA's.Decisions worth reviewing
Vendor is a new field, not an
AppleSiliconGenvariant. Twelve call sites across the VLM loaders,models/sanitize.rsanddrafter/dflash/drafter.rsreadsilicon_gen != Unknownas "this is not Apple Silicon" to gate a bf16 to f16 weight conversion. AnAmdvariant would have flipped all twelve and enabled that conversion on a backend where bf16 is native. They now callis_apple_silicon(), whose body is the identical comparison, andgpu_vendor_does_not_imply_apple_siliconfails if a future vendor ever makes it true.The vendor comes from the resolved backend, not from which
device_info()keys exist. Every backend publishesarchitecture, each in its own vocabulary:gfx1151on ROCm,sm_89on CUDA, an Apple GPU family string on Metal. Deciding from key presence would say "AMD is CUDA", which is the same shape of defect #1803 removed one layer down.gpu_backend_kind()exposesmlxcel::GpuKernelBackendto Rust and everything reads that.rocm_archis a new module, not a reuse ofcuda_arch. CUDA coverage is an ordering: a cubin runs on a higher minor revision and PTX JITs forward across majors, so a list can cover a device it does not name. HIP has neither property, so coverage is set membership over normalizedgfxtargets. Modelling it withcuda_arch'sArchVariantladder would invent a compatibility rule the runtime does not have.A generic target makes the check decline rather than refuse. ROCm 6.3 added
gfx11-generic-style targets whose code objects do load on several specific targets, so equality would report "not covered" for a build that runs. Any list this module cannot judge reports nothing, the same direction an unrecorded list takes: a wrong refusal is a binary that will not run at all, a missed one is the opaque HIP error this check was going to improve on.resolve_available_memoryis untouched in behavior. The order is now a pure function with tests, including one asserting the allocator cap is not read behind an env limit, because reading it forces the MLX allocator singleton andinspectandserve --estimate-memoryestimate before runtime bring-up on purpose.unified_memory_gbstays Apple-only so the CUDA and ROCm estimate cannot move; device memory is a separate field.hardware_label()is deliberately left alone. It collapses toUnknown-0con both CUDA and ROCm, so those two share an MTP policy cache key and a profile measured on one is reused on the other. That is a real defect, but the label is persisted in the hint files and any new spelling discards every profile recorded under the old one, which the "CUDA unchanged" constraint forbids. The vendor, device name and architecture are reported beside it in the/v1/internal/mtp-policybody instead (new fields, no schema bump). Worth its own issue.Validation
Run on the gfx1151 spike host (Radeon 8060S, ROCm 10.0.0). Both tests #1805 names pass, run directly because the full gate run still aborts on the nvfp4 case in #1806 before printing its failure list:
cargo run -p mlxcel-core --features rocm --example device_memory_probereproduces the issue's measurement table from source, with the misreport gone:rocm_arch(9),hardware(33),memory_estimate(49),streams,cuda_arch(26), the two runtime agreement tests and the mtp-policy routes (65) all pass. fmt and clippy are clean.Not verified
The cross-architecture refusal is covered by unit tests and by a test asserting the refusal tracks
rocm_arch_mismatch(), but not by a binary actually built for anothergfxtarget: that needs a full MLX rebuild in each direction. Say the word and I will run it.The full
make verify-test-rocmrun was killed twice by the host's memory watchdog while executing the 8499-testmlxcellib binary, so the targets were run individually. That is a host limit, not a regression from this branch.Gate state on this branch
Four targets still fail, all pre-existing and none introduced here. Each is now traced to a cause and filed:
mlxcel-core --lib: the nvfp4 abort of feat(quant): backend quantization capability table and load-time convert-or-reject policy #1806,Unsupported group_size for QuantizedMatmul: 16, incompiled_qgelu_mlp_global_scale_native_nvfp4_prefill_matches_reference. This is the abort that hides feat(rocm): memory estimation, device info and hardware detection on AMD UMA hosts #1805's own two failures from the gate output, which is why they are run directly above.mlxcel --test sampling_gumbel_kill_switchand--test sampling_rejection_kill_switch:[metal_kernel] No Metal back-end.The dispatch atsrc/lib/mlx-cpp/turbo/sampling.cpp:413andsampling_rejection.cpp:780branches ongpu_kernel_backend() == Cuda, so the false arm still means Metal rather than "no port for this backend". The routing gate is correct (custom_kernels_available()is false on ROCm, so production sampling takes the categorical fallback); it is the direct entry point the tests call that is unguarded.mlxcel --lib: 5 WebUI catalog tests.runnable_on_backendatsrc/server/webui/catalog_metadata.rs:309resolves toBackendStatus::Unsupportedwhenever the build is neither macOS+Metal nor CUDA, so the catalog reports that no model on this host can run. That is the same two-backend assumption this PR removes in hardware detection, surfacing one layer higher.Apart from those, the
mlxcellib target passes 8350 of 8355.