Generic numeric debugging (#19317)

[metascroy]

Summary: Pull Request resolved: #19317

Differential Revision: D103956056

Pulled By: metascroy

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/19317

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Run pull jobs on OSDC in pull requests shadow mode

❌ 4 New Failures, 2 Unrelated Failures

As of commit cc5b5a0 with merge base bd5752a ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / macos / macos-job (gh)
  exir/emit/test/test_emit.py::TestEmit::test_delegate_deduplicate
• pull / unittest / windows / windows-job (gh)
  Process completed with exit code 2.
• pull / unittest-editable / macos / macos-job (gh)
  exir/tests/test_joint_graph.py::TestJointGraph::test_joint_graph
• pull / unittest-editable / windows / windows-job (gh)
  Process completed with exit code 2.

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / unittest / linux / linux-job (gh) (detected as infra flaky with no log or failing log classifier)

BROKEN TRUNK - The following job failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  exir/tests/test_memory_planning.py::TestMisc::test_multiple_pools_1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D103956056.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[meta-codesync]

@metascroy has imported this pull request. If you are a Meta employee, you can view this in D103956056.

[digantdesai]

what happens if the delegate have to fuse some candidate nodes? Try lowering conv2d-->batch_norm, where XNNPACK doesn't support standalone batch_norm IIRC.
Also how is it better than forced-single-op-partitions?

[metascroy]

This is primarily to help RL in a numeric debugging investigation. I may clean it up to make it a generic utility if they find it useful, but will likely go through design review to get input from others if I do that.

To answer your question on fused candidates: this is tested with CoreML's quantized linear pattern [dequantize -> linear]. In that case, we tap the intermediate output after linear node, which is actually a quantized linear in both eager (from QDQ pattern) and CoreML (from its internal fusion). In the [conv2d-->batch_norm] case, I'd have to check. Tapping batch norm should mean we want the output of batch_norm, i.e., the intermediate output after batchnorm, which can be the result of a fused [conv2d-->batch_norm] op. If we did forced single-op partitions on con2d and batch_norm separately, we wouldn't get the fusion.

how is it better than forced-single-op-partitions

One reason is single-op partitions destroy fusion that backends do. Here we tap intermediates, so as long as we tap the final intermediate after a fusion pattern we should be good.

A second reason is this approach does keep the same big delegate blob, just with extra outputs. In CoreML's case that means it will still be routed to ANE, whereas if you break up into single op partitions, it will very likely reroute to CPU b/c they are small.

[YIWENX14]

LGTM.