ObjectFifo Matmul Sprint

[Abhishek-Varma]

29th July update

• We are able to compile larger shapes for i32 (Add pass to combine strided ops like NPU DMA ops #606). @jtuyls See ObjectFifo flow Matmul sweep #556 for i32 sweep
• For bf16 and i8 as well we're able to support various shapes. @Abhishek-Varma
• For Matmul+Elementwise manually modified IR passed LowerToAIEPass. @yzhang93

Plan for next day : Working on fixes for Matmul+Elementwise + triage Ukernel leading to all zeroes.

30th July update

• Raised PR for DmaLoopSubsumption ([DmaLoopSubsumption] Add support for bias in affine map + add size/stride range checks #616). @jtuyls
• Triaged the Ukernel resulting in NaNs - 256x256x256 bf16 Matmul via Ukernel works. @Abhishek-Varma
• Raised PR for inserting temporary buffers for Matmul + Elementwise ([ObjectFifo] Add InsertTemporaryBuffer Pass #615). @yzhang93

31st July update

• Raised a PR for generating 32x32x32 ukernel intrinsics + new signature + the corresponding logic in lower-to-ukernel ([ObjectFifo][Ukernel] Update ukernel signature in mm.cc and lower-to-ukernel pass #624). @Abhishek-Varma
• Addressed the review comments and merged PR ([ObjectFifo] Add InsertTemporaryBuffer Pass #615). Looked into the issue happened for small matmul-elementwise shapes Bufferization issue with redundant allocation and copy (part 2) iree-org/iree#18075. @yzhang93

1st Aug update

• Started to write a separate pass to flatten the logical objectfifo instead of doing it during LowerToAIE, so that @jtuyls could take the output from the pass and make the matmul-elementwise work through the pipeline. @yzhang93
• Tried to triage bf16 Matmul - saw similar incorrect results for i32 as well - relayed that to @jtuyls ; Jorn helped to find a way to narrow it down to compiler vs runtime issue - will be trying the same for bf16 today (2nd Aug). @Abhishek-Varma

2nd Aug update

• Verified bf16 Matmul Objectfifo to work without any issue via the pure XRT path. Also, raised [MLIR][Linalg] Fix DataLayoutPropagation for tensor.unpack + linalg.generic llvm/llvm-project#101755. @Abhishek-Varma

5th Aug update

• Verified DataLayoutPropagation changes to solve the redundant copy issue for Matmul+Elem. Started taking a look at the temporary buffer issue in DistributeCoresAndObjectFifos pass. @Abhishek-Varma

6th Aug update

• Raised temporary L1 buffer for Matmul+Elemwise ([AMD-AIE][ObjectFifo] Address temporary L1 buffers for Matmul+Elem #647) + raised PR for more bf16,i8 and bf16 ukernels CI e2e test ([ObjectFifo] Add more e2e CI test for bf16, i8 and bf16_ukernel #649). @Abhishek-Varma

7th Aug update

• Triaged bug for bf16 Matmul on objectfifo for the access op reordering ([DistributeCoresAndObjectFifos] Fix access op ordering in cores #625) and started working on splitting L2 buffers. @Abhishek-Varma

8th Aug update

• Created a pass --iree-amdaie-split-buffers (name will change) and able to get through lower-to-aie pass. @Abhishek-Varma

9th Aug update

• Adapted the code for --iree-amdaie-split-buffers pass to the changed infra in objectfifo and made the lit test exhaustive ([ObjectFifo] Create a new pass to split L2 buffers #659). @Abhishek-Varma

[MaheshRavishankar]

Thanks folks for keeping the updates coming!

[Abhishek-Varma]

This sprint we've achieved Matmul codegen working through ObjectFifo backend for i32 / bf16 / i8 / bf16 ukernel running successfully without any hang/numerical issues. And have started to made good progress on Matmul + Elemwise.

Closing this tracker now.