Skip to content

fix[next-dace]: Use SortedSet to achieve deterministic dace auto optimize#2635

Open
edopao wants to merge 8 commits into
GridTools:mainfrom
edopao:dev-toolchain_deterministic
Open

fix[next-dace]: Use SortedSet to achieve deterministic dace auto optimize#2635
edopao wants to merge 8 commits into
GridTools:mainfrom
edopao:dev-toolchain_deterministic

Conversation

@edopao

@edopao edopao commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

This is the GT4Py counterpart of dace PR spcl/dace#2398. It uses ordered data structures and unique node IDs to ensure deterministic lowering to SDFG.

This change was originally implemented by @tehrengruber in #2568.

Comment thread pyproject.toml Outdated

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR aims to make DaCe auto_optimize / lowering in the GT4Py “next” backend deterministic by eliminating order-dependence (e.g., set iteration) and replacing unstable name generation with a shared, deterministic ID generator.

Changes:

  • Switch several order-sensitive collections from set to OrderedSet to stabilize traversal/processing order.
  • Introduce and thread a shared gtx_utils.IDGeneratorPool through key DaCe transformations to generate deterministic, unique names.
  • Pin DaCe to a git branch (dev-sorted_sets) and add ordered-set as a runtime dependency to support deterministic ordered data structures.

Reviewed changes

Copilot reviewed 15 out of 16 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
uv.lock Pins dace to a specific git commit/branch and adds ordered-set to the resolved environment.
pyproject.toml Adds runtime dependency on ordered-set and configures dace as a git source.
tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/transformation_tests/test_move_tasklet_into_map.py Updates tests to pass a shared UID pool into the transformation.
tests/next_tests/unit_tests/program_processor_tests/runners_tests/dace_tests/transformation_tests/test_fuse_horizontal_conditionblocks.py Updates tests to pass a shared UID pool into the transformation.
src/gt4py/next/program_processors/runners/dace/transformations/utils.py Uses OrderedSet for deterministic upstream traversal results.
src/gt4py/next/program_processors/runners/dace/transformations/splitting_tools.py Replaces unordered edge assignment sets with OrderedSet for deterministic splitting.
src/gt4py/next/program_processors/runners/dace/transformations/split_access_nodes.py Uses OrderedSet for deterministic edge reassignment collection.
src/gt4py/next/program_processors/runners/dace/transformations/simplify.py Replaces UUID-based naming with deterministic UID pool; orders dependencies deterministically.
src/gt4py/next/program_processors/runners/dace/transformations/multi_state_global_self_copy_elimination.py Makes pass dependencies ordered (list instead of set).
src/gt4py/next/program_processors/runners/dace/transformations/move_dataflow_into_if_body.py Updates dataflow classification typing to use OrderedSet.
src/gt4py/next/program_processors/runners/dace/transformations/map_fusion_utils.py Makes connector collections and map parameter handling more deterministic.
src/gt4py/next/program_processors/runners/dace/transformations/inline_fuser.py Updates inlining logic to use OrderedSet for deterministic node ordering.
src/gt4py/next/program_processors/runners/dace/transformations/fuse_horizontal_conditionblocks.py Uses the shared UID pool for deterministic renaming during fusion.
src/gt4py/next/program_processors/runners/dace/transformations/auto_optimize.py Creates a UID pool per optimization run and threads it through key transformations.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_to_sdfg.py Stabilizes nested SDFG connector ordering via sorted connector names.
src/gt4py/next/program_processors/runners/dace/lowering/gtir_dataflow.py Stabilizes nested SDFG connector ordering (and adjusts connector representation).

Comment thread pyproject.toml

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 15 out of 16 changed files in this pull request and generated 3 comments.

Comment thread pyproject.toml
@edopao edopao changed the title fix[next-dace]: Deterministic dace auto optimize fix[next-dace]: Ise SortedSet for deterministic dace auto optimize Jun 8, 2026
@edopao edopao changed the title fix[next-dace]: Ise SortedSet for deterministic dace auto optimize fix[next-dace]: Use SortedSet to achieve deterministic dace auto optimize Jun 8, 2026
@edopao edopao marked this pull request as ready for review June 9, 2026 07:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants