Skip to content

Improve compiler driver functionality in the frontend [3/3] - #3406

Merged
dime10 merged 9 commits into
mainfrom
improve-compiler-driver-stages-3
Oct 28, 2026
Merged

dime10 merged 9 commits into
mainfrom
improve-compiler-driver-stages-3

Conversation

@dime10

@dime10 dime10 commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

This is PR 3 of 3 to improve and fix the Catalyst CLI interface pertaining to stage selection and output production. It also builds on PR #3391 to provide an accessible way of running compilation to a particular stage for testing and benchmarking.

In this PR, the target selection feature for AOT compilation is fixed to properly select the appropriate stage, even when that stage is not controlled from the frontend (like jaxpr/mlir generation or linking is). In particular:

Additional, some cleanup on previous changes in the series is performed:

  • The obsolete second "llvm_ir" return value from internal compile functions is removed, such that there is always exactly one result.
  • In order to prevent invalid state from being returned by the .llvmir accessor, the cache attribute .llvm_ir is removed from the class. While this means the textual llvm ir needs to be regenerated upon each inspection, this is inline with the other IR accessors (.mlir and .mlir_opt) and leaves the door open to overhaul these together in future work.

assisted by Astra

@codecov

codecov Bot commented Oct 27, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.24%. Comparing base (8ad0b86) to head (ebe4025).

Files with missing lines Patch % Lines
frontend/catalyst/compiler.py 91.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3406   +/-   ##
=======================================
  Coverage   96.23%   96.24%           
=======================================
  Files         177      177           
  Lines       21630    21634    +4     
  Branches     2164     2162    -2     
=======================================
+ Hits        20816    20821    +5     
  Misses        629      629           
+ Partials      185      184    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread frontend/catalyst/jit.py Resolved
Comment thread frontend/catalyst/jit.py Resolved
@dime10
dime10 force-pushed the improve-compiler-driver-stages-2 branch from 4b429c8 to 1871751 Compare October 28, 2026 17:53
@dime10
dime10 force-pushed the improve-compiler-driver-stages-3 branch from 5e076ba to 1d5ccb8 Compare October 28, 2026 17:57

@jaumevillasante jaumevillasante left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much @dime10 !

Comment thread frontend/catalyst/jit.py Resolved
Base automatically changed from improve-compiler-driver-stages-2 to main October 28, 2026 20:33
Now that the CLI exposes core compiler stages individually,
we can update the `target` selection for AOT compilation to expose
these stages to the user. The new stages include:
- llvmir_opt (stop after llvm passes are run)
- object (stop after object code is generated)

Additionally, the "llvmir" target now stops after mlir is first
translated to llvm ir, rather than compiling to object code anyway.

`link` is preserved as an internal compile option.
`lower_to_llvm` is replaced with `compiler_tool` in order to be able to
cover all options.
The llvm_ir result was already deprecated by prior
commits, this is a cleanup such that the Compiler class always
produces precisely one result:
The final output of the compilation, regardless of whether it was
partial or complete. The result value is the path to the output
artifact.
... since it may not represent the current compilation state.

This brings the `.llvmir` member inline with `.mlir` and `.mlir_opt`
which always generate their output from the current mlir_module.

Proper caching and overhaul of these accessors is thus left to future
work.
This is inline with the .mlir / .mlir_opt split.
.llvmir_opt could be added in the future if desired.
@dime10
dime10 force-pushed the improve-compiler-driver-stages-3 branch from 447537c to ebe4025 Compare October 28, 2026 20:40
@dime10
dime10 merged commit 4c83b48 into main Oct 28, 2026
38 checks passed
@dime10
dime10 deleted the improve-compiler-driver-stages-3 branch October 28, 2026 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants