Merge asplos into main

.gitignore

@@ -5,6 +5,7 @@
 /cmakeModules*
 .vscode
 __pycache__
+.DS_Store
 
 /platforms/vck190_bare/petalinux/build
 /platforms/vck190_bare/petalinux/components

README.md

@@ -14,7 +14,7 @@
 
 This repository contains an [MLIR-based](https://mlir.llvm.org/) toolchain for AI Engine-enabled devices, such as [AMD Ryzen™ AI](https://www.amd.com/en/products/ryzen-ai) and [Versal™](https://www.xilinx.com/products/technology/ai-engine.html).  This repository can be used to generate low-level configurations for the AI Engine portion of these devices. AI Engines are organized as a spatial array of tiles, where each tile contains AI Engine cores and/or memories. The spatial array is connected by stream switches that can be configured to route data between AI Engine tiles scheduled by their programmable Data Movement Accelerators (DMAs). This repository contains MLIR representations, with multiple levels of abstraction, to target AI Engine devices. This enables compilers and developers to program AI Engine cores, as well as describe data movements and array connectivity. A Python API is made available as a convenient interface for generating MLIR design descriptions. Backend code generation is also included, targeting the [aie-rt](https://github.com/Xilinx/aie-rt/tree/main-aie) library.  This toolchain uses the AI Engine compiler tool which is part of the AMD Vitis™ software installation: these tools require a free license for use from the [Product Licensing Site](https://www.xilinx.com/member/forms/license-form.html).
 
-This project is primarily intended to support the open-source community, particularly tool builders, with low-level access to AIE devices and enable the development of a wide variety of programming models from higher level abstractions.  As such, although it contains some examples, this project is not intended to represent an end-to-end compilation flow for application design. If you're looking for an out-of-the-box experience for highly efficient machine learning, check out the [AMD Ryzen™ AI Software Platform](https://github.com/amd/RyzenAI-SW/).
+This project is primarily intended to support the open-source community, particularly tool builders, with low-level access to AIE devices and enable the development of a wide variety of programming models from higher level abstractions. We provide an example programming flow: Interface Representation for hands-ON (IRON) close-to-metal programming of the AIE-array. IRON is an open access toolkit enabling performance engineers to build fast and efficient, often specialized designs through a set of Python language bindings around the mlir-aie dialect. As such, it contains some examples, however this project is not intended to represent an end-to-end compilation flow for all application designs. If you're looking for an out-of-the-box experience for highly efficient machine learning, check out the [AMD Ryzen™ AI Software Platform](https://github.com/amd/RyzenAI-SW/).
 
 [Getting Started on a Versal™ board](docs/Building.md)
 
@@ -24,8 +24,10 @@ This project is primarily intended to support the open-source community, particu
 
 [Getting Started and Running on Linux Ryzen™ AI](docs/buildHostLin.md)
 
-[Full Documentation](https://xilinx.github.io/mlir-aie/)
+[IRON AIE Application Programming Guide](programming_guide)
+
+[MLIR Dialect and Compiler Documentation](https://xilinx.github.io/mlir-aie/)
 
 -----
 
-<p align="center">Copyright&copy; 2019-2023 Advanced Micro Devices, Inc</p>
+<p align="center">Copyright&copy; 2019-2024 Advanced Micro Devices, Inc</p>

aie_kernels/aie2/scale.cc

@@ -15,6 +15,7 @@
 
 #include <aie_api/aie.hpp>
 
+// Scalar scale template
 template <typename T>
 void scale_scalar(T *a, T *c, T factor, const int32_t N) {
   event0();
@@ -24,35 +25,70 @@ void scale_scalar(T *a, T *c, T factor, const int32_t N) {
   event1();
 }
 
+// Vectorized scale template
 // Assume N is multiple of 16
 template <typename T>
-void scale_vectorized(T *a, T *c, T factor, const int32_t N) {
-  constexpr int vec_factor = 16;
+void scale_vectorized(T *a, T *c, int32_t factor, const int32_t N) {
   event0();
+  constexpr int vec_factor = 32;
   T *__restrict pA1 = a;
   T *__restrict pC1 = c;
   const int F = N / vec_factor;
+  T fac = factor;
   for (int i = 0; i < F; i++)
     chess_prepare_for_pipelining chess_loop_range(16, ) {
       aie::vector<T, vec_factor> A0 = aie::load_v<vec_factor>(pA1);
       pA1 += vec_factor;
+      aie::accum<acc32, vec_factor> cout = aie::mul(A0, fac);
+      aie::store_v(pC1, cout.template to_vector<T>(0));
+      pC1 += vec_factor;
+    }
+  event1();
+}
+
+// Vectorized scale tempalte for int32_t (acc64 used)
+// Assume N is multiple of 16
+template <>
+void scale_vectorized<int32_t>(int32_t *a, int32_t *c, int32_t factor,
+                               const int32_t N) {
+  event0();
+  constexpr int vec_factor = 32;
+  int32_t *__restrict pA1 = a;
+  int32_t *__restrict pC1 = c;
+  const int F = N / vec_factor;
+  for (int i = 0; i < F; i++)
+    chess_prepare_for_pipelining chess_loop_range(16, ) {
+      aie::vector<int32_t, vec_factor> A0 = aie::load_v<vec_factor>(pA1);
+      pA1 += vec_factor;
       aie::accum<acc64, vec_factor> cout = aie::mul(A0, factor);
-      aie::store_v(pC1, cout.to_vector<T>(0));
+      aie::store_v(pC1, cout.template to_vector<int32_t>(0));
       pC1 += vec_factor;
     }
   event1();
 }
 
 extern "C" {
 
-void vector_scalar_mul_aie(int32_t *a_in, int32_t *c_out, int32_t *factor,
-                           int32_t N) {
+// 16-bit datatype
+void vector_scalar_mul_int32_scalar(int32_t *a_in, int32_t *c_out,
+                                    int32_t *factor, int32_t N) {
+  scale_scalar<int32_t>(a_in, c_out, *factor, N);
+}
+
+void vector_scalar_mul_int32_vector(int32_t *a_in, int32_t *c_out,
+                                    int32_t *factor, int32_t N) {
   scale_vectorized<int32_t>(a_in, c_out, *factor, N);
 }
 
-void vector_scalar_mul_aie_scalar(int32_t *a_in, int32_t *c_out,
-                                  int32_t *factor, int32_t N) {
-  scale_scalar<int32_t>(a_in, c_out, *factor, N);
+// 32-bit datatype
+void vector_scalar_mul_int16_scalar(int16_t *a_in, int16_t *c_out,
+                                    int32_t *factor, int32_t N) {
+  scale_scalar<int16_t>(a_in, c_out, *factor, N);
+}
+
+void vector_scalar_mul_int16_vector(int16_t *a_in, int16_t *c_out,
+                                    int32_t *factor, int32_t N) {
+  scale_vectorized<int16_t>(a_in, c_out, *factor, N);
 }
 
 } // extern "C"

docs/buildHostLin.md

@@ -1,11 +1,30 @@
 # Linux Setup and Build Instructions
 
-These instructions will guide you through everything required for building and executing a program on the Ryzen AI NPU, starting from a fresh bare-bones **Ubuntu 22.04 LTS** install. Only Ubuntu 22.04 LTS is supported. The instructions were tested on a ASUS Vivobook Pro 15. 
+These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU, starting from a fresh bare-bones **Ubuntu 22.04 LTS** install. Only Ubuntu 22.04 LTS is supported. 
+
+## Initial Setup
+
+#### Update BIOS:
+
+Be sure you have the latest BIOS for your laptop or mini PC, this will ensure the NPU (sometimes referred to as IPU) is enabled in the system. You may need to manually enable the NPU:
+:
+   ```Advanced → CPU Configuration → IPU``` 
+
+> **NOTE:** Some manufacturers only provide Windows executables to update the BIOS, please do this before installing Ubuntu. 
+
+#### BIOS Settings:
+1. Turn off SecureBoot (Allows for unsigned drivers to be installed)
+
+   ```BIOS → Security → Secure boot → Disable```
+
+1. Turn Ac Power Loss to "Always On" (Can be used for PDU reset, turns computer back on after power loss)
+
+   ```BIOS → Advanced → AMD CBS →  FCH Common Options → Ac Power Loss Options → Set Ac Power Loss to "Always On"```
 
 ## Overview
 You will...
 
-1. Install a driver for the Ryzen AI. As part of this, you will need to...
+1. Install a driver for the Ryzen™ AI. As part of this, you will need to...
 
    1. [...compile and install a more recent Linux kernel.](#update-linux)
 
@@ -15,7 +34,7 @@ You will...
 
    1. [...install Xilinx Vitis and obtain a license.](#install-xilinx-vitis-20232-and-other-mlir-aie-prerequisites)
 
-   1. ...install MLIR-AIE [from precompiled binaries (fast)](#option-a---quick-setup-for-ryzen-ai-application-development) or [from source (slow)](#option-b---build-mlir-aie-tools-from-source-for-development).
+   1. ...install mlir-aie [from precompiled binaries (fast)](#option-a---quick-setup-for-ryzen-ai-application-development) or [from source (slow)](#option-b---build-mlir-aie-tools-from-source-for-development).
 
 1. Build and execute one of the example designs. This consists of...
 
@@ -25,7 +44,7 @@ You will...
 
    3. [...building and executing host (x86) code and device (NPU) code.](#build-and-run-host-part) 
 
-> Be advised that two of the steps (Linux compilation and Vitis install) may take hours. If you decide to build MLIR-AIE from source, this will also take a long time as it contains an LLVM build. Allocate enough time and patience. Once done, you will have an amazing toolchain allowing you to harness this great hardware at your hands.
+> Be advised that two of the steps (Linux compilation and Vitis install) may take hours. If you decide to build mlir-aie from source, this will also take a long time as it contains an LLVM build. Allocate enough time and patience. Once done, you will have an amazing toolchain allowing you to harness this great hardware at your hands.
 
 ## Prerequisites
 
@@ -203,7 +222,7 @@ You will...
    >  [0000:66:00.1]  :  RyzenAI-Phoenix 
    >  ```
 
-### Install Xilinx Vitis 2023.2 and Other MLIR-AIE Prerequisites
+### Install Xilinx Vitis 2023.2 and Other mlir-aie Prerequisites
 
 1. Install Vitis under from [Xilinx Downloads](https://www.xilinx.com/support/download/index.html/content/xilinx/en/downloadNav/vitis.html). You will need to run the installer as root. We will assume you use the default installation directory, `/tools/Xilinx`.
 
@@ -225,17 +244,17 @@ You will...
        export LM_LICENSE_FILE=/opt/Xilinx.lic
        ```
 
-1. Install the following packages needed for building MLIR-AIE:
+1. Install the following packages needed for building mlir-aie:
     ``` 
     sudo apt install \
     build-essential clang clang-14 lld lld-14 cmake python3-venv python3-pip libxrender1 libxtst6 libxi6
       ```
 
-1. Choose *one* of the two options (A or B) below for installing MLIR-AIE.
+1. Choose *one* of the two options (A or B) below for installing mlir-aie.
 
-### Option A - Quick Setup for Ryzen AI Application Development
+### Option A - Quick Setup for Ryzen™ AI Application Development
 
-1. Clone [the MLIR-AIE repository](https://github.com/Xilinx/mlir-aie.git), best under /home/username for speed (yourPathToBuildMLIR-AIE): 
+1. Clone [the mlir-aie repository](https://github.com/Xilinx/mlir-aie.git), best under /home/username for speed (yourPathToBuildMLIR-AIE): 
    ```
    git clone https://github.com/Xilinx/mlir-aie.git
    cd mlir-aie
@@ -246,7 +265,7 @@ You will...
 
 1. Jump ahead to [Build Device AIE Part](#build-device-aie-part) step 2 below.
 
-### Option B - Build MLIR-AIE Tools from Source for Development
+### Option B - Build mlir-aie Tools from Source for Development
 
 1. Clone [https://github.com/Xilinx/mlir-aie.git](https://github.com/Xilinx/mlir-aie.git) best under /home/username for speed (yourPathToBuildMLIR-AIE), with submodules: 
    ```
@@ -275,7 +294,7 @@ source ${MLIR_AIE_BUILD_DIR}/ironenv/bin/activate
 source ${MLIR_AIE_BUILD_DIR}/utils/env_setup.sh ${MLIR_AIE_BUILD_DIR}/my_install/mlir_aie ${MLIR_AIE_BUILD_DIR}/my_install/mlir
 ```
 
-> Replace `${MLIR_AIE_BUILD_DIR}` with the directory in which you *built* MLIR-AIE above. Replace `${NEW_CMAKE_DIR}` with the directory in which you installed CMake 3.28 above. Instead of search and replace, you can also define these values as environment variables.
+> Replace `${MLIR_AIE_BUILD_DIR}` with the directory in which you *built* mlir-aie above. Replace `${NEW_CMAKE_DIR}` with the directory in which you installed CMake 3.28 above. Instead of search and replace, you can also define these values as environment variables.
 
 > For quick setup, this step is only needed if you are starting with a new terminal. If you are continuing in the same terminal you used to install the prerequisites, the environment variables should all be set.
 
@@ -289,15 +308,15 @@ source /opt/xilinx/xrt/setup.sh
 source ${MLIR_AIE_BUILD_DIR}/utils/env_setup.sh ${MLIR_AIE_BUILD_DIR}/install ${MLIR_AIE_BUILD_DIR}/llvm/install
 ```
 
-> Replace `${MLIR_AIE_BUILD_DIR}` with the directory in which you *built* MLIR-AIE above. Instead of search and replace, you can also define `MLIR_AIE_BUILD_DIR` as an environment variable.
+> Replace `${MLIR_AIE_BUILD_DIR}` with the directory in which you *built* mlir-aie above. Instead of search and replace, you can also define `MLIR_AIE_BUILD_DIR` as an environment variable.
 
 ## Build a Design
 
 For your design of interest, for instance [vector_add](../programming_examples/basic/vector_add/), 2 steps are needed: (i) build the AIE desgin and then (ii) build the host code.
 
 ### Build Device AIE Part
 
-1. Prepare your enviroment with the MLIR-AIE tools (built during prerequisites part of this guide) - see **"Setting Up Your Environment"** avove.
+1. Prepare your enviroment with the mlir-aie tools (built during prerequisites part of this guide) - see **"Setting Up Your Environment"** avove.
 
 2. Goto the design of interest and run `make`
 

docs/buildHostWin.md

@@ -1,14 +1,34 @@
 # Windows Setup and Build Instructions
 
-These instructions will guide you through everything required for building and executing a program on the Ryzen AI NPU on Windows. The instructions were tested on a ASUS Vivobook Pro 15. 
+These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU on Windows. The instructions were tested on a ASUS Vivobook Pro 15. 
 
 You will set up a Windows subsystem for Linux (WSL) Ubuntu install, which will be used for building NPU device code. For building the host (x86) code, you will use MS Visual Code Community.
 
-- Rely on WSL Ubuntu 22.04 LTS for Vitis tool install and to build and run our MLIR-AIE tools
+- Rely on WSL Ubuntu 22.04 LTS for Vitis tool install and to build and run our mlir-aie tools
 - Rely on MS Visual Studio 17 2022 to natively build the host code (aka test.cpp)
 
+## Initial Setup
+
+#### Update BIOS:
+
+Be sure you have the latest BIOS for your laptop or mini PC, this will ensure the NPU (sometimes referred to as IPU) is enabled in the system. You may need to manually enable the NPU:
+
+  ```Advanced → CPU Configuration → IPU```
+
+> **NOTE:** Some manufacturers only provide Windows executables to update the BIOS. 
+
+#### BIOS Settings:
+1. Turn off SecureBoot (Allows for unsigned drivers to be installed)
+
+   ```BIOS → Security → Secure boot → Disable```
+
+1. Turn Ac Power Loss to "Always On" (Can be used for PDU reset, turns computer back on after power loss)
+
+   ```BIOS → Advanced → AMD CBS →  FCH Common Options → Ac Power Loss Options → Set Ac Power Loss to "Always On"```
+
+
 ## Prerequisites
-### MLIR-AIE tools: WSL Ubuntu 22.04
+### mlir-aie tools: WSL Ubuntu 22.04
 All steps in WSL Ubuntu terminal.
 1. Clone [https://github.com/Xilinx/mlir-aie.git](https://github.com/Xilinx/mlir-aie.git) best under /home/username for speed (yourPathToBuildMLIR-AIE), with submodules: 
    ```
@@ -44,7 +64,7 @@ All steps in WSL Ubuntu terminal.
       ip link set vmnic0 addr <yourMACaddress>
       ```
 
-1. Install or Build MLIR-AIE tools under WSL2:
+1. Install or Build mlir-aie tools under WSL2:
 
    * Use quick setup script to install from whls:
      ```
@@ -56,7 +76,7 @@ All steps in WSL Ubuntu terminal.
 
    * [Optional] Build from source following regular get started instructions [https://xilinx.github.io/mlir-aie/Building.html](https://xilinx.github.io/mlir-aie/Building.html)
 
-1. After installing the updated RyzenAI driver (see next subsection), use the gendef tool (from the mingw-w64-tools package) to create a .def file with the symbols:
+1. After installing the updated Ryzen™ AI driver (see next subsection), use the gendef tool (from the mingw-w64-tools package) to create a .def file with the symbols:
     ```
     mkdir /mnt/c/Technical/xrtNPUfromDLL; cd /mnt/c/Technical/xrtNPUfromDLL
     cp /mnt/c/Windows/System32/AMD/xrt_coreutil.dll .
@@ -83,15 +103,15 @@ All steps in Win11 (powershell where needed).
       ```
       lib /def:xrt_coreutil.def /machine:x64 /out:xrt_coreutil.lib
       ```
-1. Clone [https://github.com/Xilinx/mlir-aie.git]([https://gitenterprise.xilinx.com/XRLabs/pynqMLIR-AIE](https://github.com/Xilinx/mlir-aie.git)) for instance under C:\Technical to be used to build designs (yourPathToDesignsWithMLIR-AIE) 
+1. Clone [https://github.com/Xilinx/mlir-aie.git](https://github.com/Xilinx/mlir-aie.git) for instance under C:\Technical to be used to build designs (yourPathToDesignsWithMLIR-AIE) 
 
 ## Set up your environment
 
 To make the compilation toolchain available for use in your WSL terminal, you will need to set some environment variables. We suggest you add the following to a file named `setup.sh`, so you can set up your environment easily by running `source setup.sh`.
 
 ### `setup.sh` - Option A - Using Quick Setup
 
-If you used the quick setup script (precompiled MLIR-AIE binaries), use this setup script.
+If you used the quick setup script (precompiled mlir-aie binaries), use this setup script.
 
 ```
 # NOTE: if you did NOT exit the terminal you can skip this step.