fix[next-dace]: Use SortedSet to achieve deterministic dace auto optimize#2635
Open
edopao wants to merge 8 commits into
Open
fix[next-dace]: Use SortedSet to achieve deterministic dace auto optimize#2635edopao wants to merge 8 commits into
SortedSet to achieve deterministic dace auto optimize#2635edopao wants to merge 8 commits into
Conversation
edopao
commented
Jun 8, 2026
There was a problem hiding this comment.
Pull request overview
This PR aims to make DaCe auto_optimize / lowering in the GT4Py “next” backend deterministic by eliminating order-dependence (e.g., set iteration) and replacing unstable name generation with a shared, deterministic ID generator.
Changes:
- Switch several order-sensitive collections from
settoOrderedSetto stabilize traversal/processing order. - Introduce and thread a shared
gtx_utils.IDGeneratorPoolthrough key DaCe transformations to generate deterministic, unique names. - Pin DaCe to a git branch (
dev-sorted_sets) and addordered-setas a runtime dependency to support deterministic ordered data structures.
Reviewed changes
Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
uv.lock |
Pins dace to a specific git commit/branch and adds ordered-set to the resolved environment. |
pyproject.toml |
Adds runtime dependency on ordered-set and configures dace as a git source. |
tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/transformation_tests/test_move_tasklet_into_map.py |
Updates tests to pass a shared UID pool into the transformation. |
tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/transformation_tests/test_fuse_horizontal_conditionblocks.py |
Updates tests to pass a shared UID pool into the transformation. |
src/gt4py/next/program_processors/runners/dace/transformations/utils.py |
Uses OrderedSet for deterministic upstream traversal results. |
src/gt4py/next/program_processors/runners/dace/transformations/splitting_tools.py |
Replaces unordered edge assignment sets with OrderedSet for deterministic splitting. |
src/gt4py/next/program_processors/runners/dace/transformations/split_access_nodes.py |
Uses OrderedSet for deterministic edge reassignment collection. |
src/gt4py/next/program_processors/runners/dace/transformations/simplify.py |
Replaces UUID-based naming with deterministic UID pool; orders dependencies deterministically. |
src/gt4py/next/program_processors/runners/dace/transformations/multi_state_global_self_copy_elimination.py |
Makes pass dependencies ordered (list instead of set). |
src/gt4py/next/program_processors/runners/dace/transformations/move_dataflow_into_if_body.py |
Updates dataflow classification typing to use OrderedSet. |
src/gt4py/next/program_processors/runners/dace/transformations/map_fusion_utils.py |
Makes connector collections and map parameter handling more deterministic. |
src/gt4py/next/program_processors/runners/dace/transformations/inline_fuser.py |
Updates inlining logic to use OrderedSet for deterministic node ordering. |
src/gt4py/next/program_processors/runners/dace/transformations/fuse_horizontal_conditionblocks.py |
Uses the shared UID pool for deterministic renaming during fusion. |
src/gt4py/next/program_processors/runners/dace/transformations/auto_optimize.py |
Creates a UID pool per optimization run and threads it through key transformations. |
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg.py |
Stabilizes nested SDFG connector ordering via sorted connector names. |
src/gt4py/next/program_processors/runners/dace/lowering/gtir_dataflow.py |
Stabilizes nested SDFG connector ordering (and adjusts connector representation). |
SortedSet for deterministic dace auto optimize
SortedSet for deterministic dace auto optimizeSortedSet to achieve deterministic dace auto optimize
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is the GT4Py counterpart of dace PR spcl/dace#2398. It uses ordered data structures and unique node IDs to ensure deterministic lowering to SDFG.
This change was originally implemented by @tehrengruber in #2568.