Add TagUnquantizedNodesPass

[Aleksei-grovety]

For models with operations that are not quantized, this pass keeps unquantized operators on the CPU. For example, the deit-tiny-patch16-224 network has an unquantized scaled_dot_product_attention operation. When compiling to Vela, invalid argument errors occur because unquantized operations are offloaded to the NPU. This pass is designed to solve this problem.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/7427

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit a5db1ed with merge base 82763a9 ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[Aleksei-grovety]

@pytorchbot label "topic: not user facing"

[Aleksei-grovety]

@pytorchbot label "partner: arm"

[oscarandersson8218]

Thanks for your contribution @Aleksei-grovety! Just for clarification, do you prefer ScaledDotProductAttention (SDPA) to run on the CPU in FP32 for precision reasons, or would you prefer to run it on the NPU?

If you prefer to run it on the NPU, a different approach to this would be to decompose SDPA before quantization. An SDPA-decomposition exists in https://github.com/pytorch/executorch/blob/main/backends/transforms/decompose_sdpa.py. You can add pre-quantization decompositions to

executorch/backends/arm/_passes/arm_pass_manager.py

Line 144 in e2afedf

def transform_for_annotation_pipeline(self, graph_module: torch.fx.GraphModule):

Note: doing this might require some additional changes to https://github.com/pytorch/executorch/blob/main/backends/arm/_passes/decompose_softmaxes_pass.py