Add ordered atomics and enable KernelAbstractions atomic support - #889
Merged
Conversation
Metal's low-level atomic intrinsics previously exposed only relaxed ordering. That is fine for counters and reductions, but not for synchronization patterns where ordinary device-memory writes are published through an atomic and then observed by another workitem. Thread explicit memory orderings through the low-level atomic API and light up KernelAbstractions/Atomix atomics on the Metal backend: - Add explicit `order` (and `flags`/`scope`) arguments to the low-level atomic store/load/exchange/fetch/compare-exchange intrinsics, using `Val`-specialized paths so orderings reach GPU code as compile-time constants. - Keep the existing direct call sites on relaxed ordering, so code that only needs relaxed atomics keeps emitting the same `air.atomic.*` forms and does not change behaviour. - Support independent success/failure orderings on compare-exchange. - Gate the acquire/release/acq_rel/seq_cst orderings and the memory flags on Metal 4.1 with device-side `@static_assert`s (`check_atomic_memory_order` / `check_atomic_flags`), so unavailable operations fail to compile with a clear message and enclosing `metal_version()` guards can eliminate them. - Enable `KA.supports_atomics(::MetalBackend)` when the device supports Metal 4.1, so Atomix-based kernels (e.g. KernelAbstractions' `@atomic`) work on Metal. The `memory_order` / `thread_scope` enums and `atomic_thread_fence` live in synchronization.jl as of #875; this leaves that file untouched and keeps the ordering-availability helpers next to their only users in atomics.jl. Tests cover the explicit-ordering low-level paths, the emitted IR ordering/flag constants, the Metal-4.1 availability errors, and a cross-workitem publish/consume regression ("device-memory publish through fetch_add") modelling a bottom-up refit: each workitem writes ordinary device memory then increments an ordered atomic counter, and the second arrival reads both children's writes before publishing the parent. Co-authored-by: Rémi Vezy <VEZY@users.noreply.github.com> Co-authored-by: Christian Guinard <28689358+christiangnrd@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Closed
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #889 +/- ##
==========================================
+ Coverage 85.93% 86.26% +0.33%
==========================================
Files 76 76
Lines 5246 5300 +54
==========================================
+ Hits 4508 4572 +64
+ Misses 738 728 -10 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
There was a problem hiding this comment.
Metal Benchmarks
Details
| Benchmark suite | Current: 7b392d3 | Previous: e527abe | Ratio |
|---|---|---|---|
array/accumulate/Float32/1d |
406042 ns |
419167 ns |
0.97 |
array/accumulate/Float32/dims=1 |
376458 ns |
396959 ns |
0.95 |
array/accumulate/Float32/dims=1L |
8807208 ns |
8882417 ns |
0.99 |
array/accumulate/Float32/dims=2 |
446458 ns |
455959 ns |
0.98 |
array/accumulate/Float32/dims=2L |
2594792 ns |
2288250 ns |
1.13 |
array/accumulate/Int64/1d |
849167 ns |
873792 ns |
0.97 |
array/accumulate/Int64/dims=1 |
933750 ns |
953833 ns |
0.98 |
array/accumulate/Int64/dims=1L |
9533416 ns |
9673416 ns |
0.99 |
array/accumulate/Int64/dims=2 |
1252500 ns |
969791 ns |
1.29 |
array/accumulate/Int64/dims=2L |
6526667 ns |
6737834 ns |
0.97 |
array/broadcast |
221125 ns |
187875 ns |
1.18 |
array/construct |
2375 ns |
2458 ns |
0.97 |
array/permutedims/2d |
453500 ns |
431375 ns |
1.05 |
array/permutedims/3d |
1018083 ns |
729667 ns |
1.40 |
array/permutedims/4d |
1142709 ns |
1054333 ns |
1.08 |
array/private/copy |
236291 ns |
227500 ns |
1.04 |
array/private/copyto!/cpu_to_gpu |
195667 ns |
204500 ns |
0.96 |
array/private/copyto!/gpu_to_cpu |
190792 ns |
203666 ns |
0.94 |
array/private/copyto!/gpu_to_gpu |
204208 ns |
185917 ns |
1.10 |
array/private/iteration/findall/bool |
1059125 ns |
1109917 ns |
0.95 |
array/private/iteration/findall/int |
1236625 ns |
1243916 ns |
0.99 |
array/private/iteration/findfirst/bool |
1128625 ns |
1088208 ns |
1.04 |
array/private/iteration/findfirst/int |
857917 ns |
1118125 ns |
0.77 |
array/private/iteration/findmin/1d |
1230166 ns |
1270125 ns |
0.97 |
array/private/iteration/findmin/2d |
1079666 ns |
1056416 ns |
1.02 |
array/private/iteration/logical |
1547750 ns |
1853875 ns |
0.83 |
array/private/iteration/scalar |
1231458 ns |
1160458 ns |
1.06 |
array/random/rand/Float32 |
435542 ns |
406667 ns |
1.07 |
array/random/rand/Int64 |
521459 ns |
544667 ns |
0.96 |
array/random/rand!/Float32 |
392291 ns |
373875 ns |
1.05 |
array/random/rand!/Int64 |
409292 ns |
404125 ns |
1.01 |
array/random/randn/Float32 |
392041 ns |
395709 ns |
0.99 |
array/random/randn!/Float32 |
351625 ns |
331625 ns |
1.06 |
array/reductions/mapreduce/Float32/1d |
383791 ns |
385083 ns |
1.00 |
array/reductions/mapreduce/Float32/dims=1 |
339458 ns |
325667 ns |
1.04 |
array/reductions/mapreduce/Float32/dims=1L |
626708 ns |
631583 ns |
0.99 |
array/reductions/mapreduce/Float32/dims=2 |
344750 ns |
329250 ns |
1.05 |
array/reductions/mapreduce/Float32/dims=2L |
998083 ns |
778875 ns |
1.28 |
array/reductions/mapreduce/Int64/1d |
585875 ns |
583000 ns |
1.00 |
array/reductions/mapreduce/Int64/dims=1 |
621792 ns |
455542 ns |
1.36 |
array/reductions/mapreduce/Int64/dims=1L |
1035042 ns |
1048250 ns |
0.99 |
array/reductions/mapreduce/Int64/dims=2 |
782250 ns |
516750 ns |
1.51 |
array/reductions/mapreduce/Int64/dims=2L |
2188333 ns |
2170125 ns |
1.01 |
array/reductions/reduce/Float32/1d |
385375 ns |
390083 ns |
0.99 |
array/reductions/reduce/Float32/dims=1 |
280958 ns |
323834 ns |
0.87 |
array/reductions/reduce/Float32/dims=1L |
613083 ns |
619167 ns |
0.99 |
array/reductions/reduce/Float32/dims=2 |
231959 ns |
198166 ns |
1.17 |
array/reductions/reduce/Float32/dims=2L |
458667 ns |
450084 ns |
1.02 |
array/reductions/reduce/Int64/1d |
615750 ns |
582542 ns |
1.06 |
array/reductions/reduce/Int64/dims=1 |
629041 ns |
554334 ns |
1.13 |
array/reductions/reduce/Int64/dims=1L |
1031292 ns |
1045250 ns |
0.99 |
array/reductions/reduce/Int64/dims=2 |
258333 ns |
241334 ns |
1.07 |
array/reductions/reduce/Int64/dims=2L |
647167 ns |
636542 ns |
1.02 |
array/shared/copy |
136875 ns |
143416 ns |
0.95 |
array/shared/copyto!/cpu_to_gpu |
37958 ns |
38000 ns |
1.00 |
array/shared/copyto!/gpu_to_cpu |
38875 ns |
38583 ns |
1.01 |
array/shared/copyto!/gpu_to_gpu |
38750 ns |
38208 ns |
1.01 |
array/shared/iteration/findall/bool |
1076667 ns |
1137958 ns |
0.95 |
array/shared/iteration/findall/int |
1251834 ns |
1270792 ns |
0.99 |
array/shared/iteration/findfirst/bool |
958667 ns |
640708 ns |
1.50 |
array/shared/iteration/findfirst/int |
962625 ns |
812541 ns |
1.18 |
array/shared/iteration/findmin/1d |
1096875 ns |
975458 ns |
1.12 |
array/shared/iteration/findmin/2d |
1070792 ns |
1081334 ns |
0.99 |
array/shared/iteration/logical |
1559209 ns |
1712625 ns |
0.91 |
array/shared/iteration/scalar |
4017.8571428571427 ns |
4035.714285714286 ns |
1.00 |
array/sorting/1d |
1787667 ns |
1853000 ns |
0.96 |
array/sorting/2d |
8282541 ns |
8321959 ns |
1.00 |
integration/byval/reference |
1103708 ns |
1099458 ns |
1.00 |
integration/byval/slices=1 |
1117209 ns |
1110709 ns |
1.01 |
integration/byval/slices=2 |
2023917 ns |
2040334 ns |
0.99 |
integration/byval/slices=3 |
6636625 ns |
15597292 ns |
0.43 |
integration/metaldevrt |
376500 ns |
376500 ns |
1 |
kernel/indexing |
196208 ns |
171208 ns |
1.15 |
kernel/indexing_checked |
384917 ns |
308250 ns |
1.25 |
kernel/launch |
1895.8 ns |
1916.7 ns |
0.99 |
kernel/rand |
386916 ns |
314875 ns |
1.23 |
latency/import |
2040973208 ns |
2068127667 ns |
0.99 |
latency/precompile |
38933808083 ns |
39359634375 ns |
0.99 |
latency/ttfp |
2380935584 ns |
2408685084 ns |
0.99 |
metal/synchronization/context |
595.2067039106146 ns |
593.3988764044943 ns |
1.00 |
metal/synchronization/stream |
332.96 ns |
331.6036866359447 ns |
1.00 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Metal's low-level atomic intrinsics previously exposed only relaxed ordering. That is fine for counters and reductions, but not for synchronization patterns where ordinary device-memory writes are published through an atomic and then observed by another workitem.
Thread explicit memory orderings through the low-level atomic API and light up KernelAbstractions/Atomix atomics on the Metal backend:
order(andflags/scope) arguments to the low-level atomic store/load/exchange/fetch/compare-exchange intrinsics, usingVal-specialized paths so orderings reach GPU code as compile-time constants.air.atomic.*forms and does not change behaviour.@static_asserts (check_atomic_memory_order/check_atomic_flags), so unavailable operations fail to compile with a clear message and enclosingmetal_version()guards can eliminate them.KA.supports_atomics(::MetalBackend)when the device supports Metal 4.1, so Atomix-based kernels (e.g. KernelAbstractions'@atomic) work on Metal.