From 00c621191eda9de071da175890d1e5a620ca44a1 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 15:41:55 +0800 Subject: [PATCH 01/17] add tutorial for inductor on windows cpu --- prototype_source/inductor_windows_cpu.rst | 73 +++++++++++++++++++++++ 1 file changed, 73 insertions(+) create mode 100644 prototype_source/inductor_windows_cpu.rst diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst new file mode 100644 index 0000000000..be0ae8eb6c --- /dev/null +++ b/prototype_source/inductor_windows_cpu.rst @@ -0,0 +1,73 @@ +How to use TorchInductor on Windows CPU +======================================= + +**Author**: `Xu, Han `_, `Zhaoqiong Zheng `_ + + +Introduction +------------ + +TorchInductor is the new compiler backend that compiles the FX Graphs generated by TorchDynamo into optimized C++/Triton kernels. + +This tutorial introduces the steps for utilizing TorchInductor on Windows CPU. + + + +Software Installation +--------------------- + +Now, we will walk you through a step-by-step tutorial for how to use torchinductor on Windows CPU. + +Install a Compiler +^^^^^^^^^^^^^^^^^^ + +TorchInductor Optimization need a compiler, let's take MSVC as an example. + +Download and install `MSVC `_. + +During Installation, chosse `Desktop Development with C++` in the `Desktop & Mobile` Section and then install. + +.. note:: + + You can also choose other compiler like Clang & Intel Compiler. + + +Install Miniforge +^^^^^^^^^^^^^^^^^ + +Download and install `Miniforge `_. + +Set Up Environment +^^^^^^^^^^^^^^^^^^ + +1. Open a command line environment via cmd.exe. +2. Activate `MSVC` via below command: +:: + + "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Auxiliary/Build/vcvars64.bat" +3. Activate `conda` via below command: +:: + + "C:/ProgramData/miniforge3/Scripts/activate.bat" +4. Create and activate customer conda environment: +:: + + conda create -n inductor_cpu_windows python=3.10 -y + conda activate inductor_cpu_windows + +5. `Install PyTorch 2.5 `_ or later. +6. Try `torchinductor` on Windows CPU: +:: + + import torch + def foo(x, y): + a = torch.sin(x) + b = torch.cos(x) + return a + b + opt_foo1 = torch.compile(foo) + print(opt_foo1(torch.randn(10, 10), torch.randn(10, 10))) + +Conclusion +---------- + +With this tutorial, we introduce how to use Inductor with X86 CPU in PyTorch 2.5 on Windows. From 10b6d4b4fcb142557ede01780ee48f7d68a6be82 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 15:56:16 +0800 Subject: [PATCH 02/17] update tutorial inductor_windows_cpu --- prototype_source/inductor_windows_cpu.rst | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index be0ae8eb6c..a8d7a8baa0 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -21,7 +21,7 @@ Now, we will walk you through a step-by-step tutorial for how to use torchinduct Install a Compiler ^^^^^^^^^^^^^^^^^^ -TorchInductor Optimization need a compiler, let's take MSVC as an example. +C++ compiler is required for torchinductor optimization, let's take Microsoft Visual C++ (MSVC) as an example. Download and install `MSVC `_. @@ -54,8 +54,7 @@ Set Up Environment conda create -n inductor_cpu_windows python=3.10 -y conda activate inductor_cpu_windows - -5. `Install PyTorch 2.5 `_ or later. +5. Install `PyTorch 2.5 `_ or later. 6. Try `torchinductor` on Windows CPU: :: @@ -70,4 +69,4 @@ Set Up Environment Conclusion ---------- -With this tutorial, we introduce how to use Inductor with X86 CPU in PyTorch 2.5 on Windows. +With this tutorial, we introduce how to use Inductor on Windows CPU with PyTorch 2.5 or later. From d1a98dfdcd926a6e594463277b0df350dc9f3566 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 17:03:10 +0800 Subject: [PATCH 03/17] update inductor windows cpu --- prototype_source/inductor_windows_cpu.rst | 42 +++++++++++++++++++---- 1 file changed, 36 insertions(+), 6 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index a8d7a8baa0..754c4f3320 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -1,7 +1,7 @@ How to use TorchInductor on Windows CPU ======================================= -**Author**: `Xu, Han `_, `Zhaoqiong Zheng `_ +**Author**: `Zhaoqiong Zheng `_, `Xu, Han `_ Introduction @@ -29,7 +29,8 @@ During Installation, chosse `Desktop Development with C++` in the `Desktop & Mob .. note:: - You can also choose other compiler like Clang & Intel Compiler. + We recommend C++ compiler `Clang `_ and `Intel Compiler`_. + Intel compiler setup guide(TODO: Intel will provide URL for this guide recently) Install Miniforge @@ -41,21 +42,26 @@ Set Up Environment ^^^^^^^^^^^^^^^^^^ 1. Open a command line environment via cmd.exe. + 2. Activate `MSVC` via below command: :: "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Auxiliary/Build/vcvars64.bat" -3. Activate `conda` via below command: + +1. Activate `conda` via below command: :: "C:/ProgramData/miniforge3/Scripts/activate.bat" -4. Create and activate customer conda environment: + +1. Create and activate customer conda environment: :: conda create -n inductor_cpu_windows python=3.10 -y conda activate inductor_cpu_windows -5. Install `PyTorch 2.5 `_ or later. -6. Try `torchinductor` on Windows CPU: + +1. Install `PyTorch 2.5 `_ or later. + +2. Try `torchinductor` on Windows CPU: :: import torch @@ -66,6 +72,30 @@ Set Up Environment opt_foo1 = torch.compile(foo) print(opt_foo1(torch.randn(10, 10), torch.randn(10, 10))) +output of the above example. +:: + + tensor([[-3.9074e-02, 1.3994e+00, 1.3894e+00, 3.2630e-01, 8.3060e-01, + 1.1833e+00, 1.4016e+00, 7.1905e-01, 9.0637e-01, -1.3648e+00], + [ 1.3728e+00, 7.2863e-01, 8.6888e-01, -6.5442e-01, 5.6790e-01, + 5.2025e-01, -1.2647e+00, 1.2684e+00, -1.2483e+00, -7.2845e-01], + [-6.7747e-01, 1.2028e+00, 1.1431e+00, 2.7196e-02, 5.5304e-01, + 6.1945e-01, 4.6654e-01, -3.7376e-01, 9.3644e-01, 1.3600e+00], + [-1.0157e-01, 7.7200e-02, 1.0146e+00, 8.8175e-02, -1.4057e+00, + 8.8119e-01, 6.2853e-01, 3.2773e-01, 8.5082e-01, 8.4615e-01], + [ 1.4140e+00, 1.2130e+00, -2.0762e-01, 3.3914e-01, 4.1122e-01, + 8.6895e-01, 5.8852e-01, 9.3310e-01, 1.4101e+00, 9.8318e-01], + [ 1.2355e+00, 7.9290e-02, 1.3707e+00, 1.3754e+00, 1.3768e+00, + 9.8970e-01, 1.1171e+00, -5.9944e-01, 1.2553e+00, 1.3394e+00], + [-1.3428e+00, 1.8400e-01, 1.1756e+00, -3.0654e-01, 9.7973e-01, + 1.4019e+00, 1.1886e+00, -1.9194e-01, 1.3632e+00, 1.1811e+00], + [-7.1615e-01, 4.6622e-01, 1.2089e+00, 9.2011e-01, 1.0659e+00, + 9.0892e-01, 1.1932e+00, 1.3888e+00, 1.3898e+00, 1.3218e+00], + [ 1.4139e+00, -1.4000e-01, 9.1192e-01, 3.0175e-01, -9.6432e-01, + -1.0498e+00, 1.4115e+00, -9.3212e-01, -9.0964e-01, 1.0127e+00], + [ 5.7244e-04, 1.2799e+00, 1.3595e+00, 1.0907e+00, 3.7191e-01, + 1.4062e+00, 1.3672e+00, 6.8502e-02, 8.5216e-01, 8.6046e-01]]) + Conclusion ---------- From 4df70a7edbcfbd846e99aef31e1620cbbb88e45b Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 17:13:46 +0800 Subject: [PATCH 04/17] update inductor windows cpu --- prototype_source/inductor_windows_cpu.rst | 29 +++++++---------------- 1 file changed, 9 insertions(+), 20 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 754c4f3320..a9f7e8daf5 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -29,8 +29,8 @@ During Installation, chosse `Desktop Development with C++` in the `Desktop & Mob .. note:: - We recommend C++ compiler `Clang `_ and `Intel Compiler`_. - Intel compiler setup guide(TODO: Intel will provide URL for this guide recently) + We recommend C++ compiler `Clang `_ and `Intel Compiler `_. + Please follow `Intel compiler setup guide `_. (TODO: Intel will provide URL for this guide recently) Install Miniforge @@ -41,29 +41,19 @@ Download and install `Miniforge `_ or later. - -2. Try `torchinductor` on Windows CPU: -:: - +#. Install `PyTorch 2.5 `_ or later. +#. Try `torchinductor` on Windows CPU:: import torch def foo(x, y): a = torch.sin(x) @@ -72,8 +62,7 @@ Set Up Environment opt_foo1 = torch.compile(foo) print(opt_foo1(torch.randn(10, 10), torch.randn(10, 10))) -output of the above example. -:: +#. Output of the above example:: tensor([[-3.9074e-02, 1.3994e+00, 1.3894e+00, 3.2630e-01, 8.3060e-01, 1.1833e+00, 1.4016e+00, 7.1905e-01, 9.0637e-01, -1.3648e+00], From c8f29b6cc7fd4b5aef328e2f07850f69d322057f Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 17:15:45 +0800 Subject: [PATCH 05/17] update inductor windows cpu and fix format --- prototype_source/inductor_windows_cpu.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index a9f7e8daf5..da06342827 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -53,7 +53,8 @@ Set Up Environment conda create -n inductor_cpu_windows python=3.10 -y conda activate inductor_cpu_windows #. Install `PyTorch 2.5 `_ or later. -#. Try `torchinductor` on Windows CPU:: +#. Use torchinductor on Windows CPU:: + import torch def foo(x, y): a = torch.sin(x) From d0c485b2d17c886d0b0e0b71975a6d9d5a7dd0a9 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Thu, 26 Sep 2024 17:37:02 +0800 Subject: [PATCH 06/17] remove icx guide --- prototype_source/inductor_windows_cpu.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index da06342827..b123a947e1 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -30,8 +30,6 @@ During Installation, chosse `Desktop Development with C++` in the `Desktop & Mob .. note:: We recommend C++ compiler `Clang `_ and `Intel Compiler `_. - Please follow `Intel compiler setup guide `_. (TODO: Intel will provide URL for this guide recently) - Install Miniforge ^^^^^^^^^^^^^^^^^ @@ -54,7 +52,7 @@ Set Up Environment conda activate inductor_cpu_windows #. Install `PyTorch 2.5 `_ or later. #. Use torchinductor on Windows CPU:: - + import torch def foo(x, y): a = torch.sin(x) From 3ea5e0567d5b263d68ef87112d471b44e44624b3 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Thu, 26 Sep 2024 09:13:25 -0700 Subject: [PATCH 07/17] Editorial and formatting fixes --- prototype_source/inductor_windows_cpu.rst | 66 ++++++++++++++--------- 1 file changed, 42 insertions(+), 24 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index b123a947e1..4d980eccab 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -4,54 +4,70 @@ How to use TorchInductor on Windows CPU **Author**: `Zhaoqiong Zheng `_, `Xu, Han `_ -Introduction ------------- -TorchInductor is the new compiler backend that compiles the FX Graphs generated by TorchDynamo into optimized C++/Triton kernels. +TorchInductor is a new compiler backend that compiles FX Graphs generated by TorchDynamo into optimized C++/Triton kernels. +This tutorial will guide you through the process of using TorchInductor on a Windows CPU. -This tutorial introduces the steps for utilizing TorchInductor on Windows CPU. +.. grid:: 2 + .. grid-item-card:: :octicon:`mortar-board;1em;` What you will learn + :class-card: card-prerequisites + * How to compile and execute a Python function with PyTorch, optimized for Windows CPU + * Basics of TorchInductor's optimization using C++/Triton kernels. -Software Installation ---------------------- + .. grid-item-card:: :octicon:`list-unordered;1em;` Prerequisites + :class-card: card-prerequisites -Now, we will walk you through a step-by-step tutorial for how to use torchinductor on Windows CPU. + * PyTorch v2.5 or later + * Microsoft Visual C++ (MSVC) + * Miniforge for Windows -Install a Compiler -^^^^^^^^^^^^^^^^^^ +Install the Required Software +--------------------- -C++ compiler is required for torchinductor optimization, let's take Microsoft Visual C++ (MSVC) as an example. +First, let's install the required software. C++ compiler is required for TorchInductor optimization. +We will use Microsoft Visual C++ (MSVC) for this example. -Download and install `MSVC `_. +1. Download and install `MSVC `_. -During Installation, chosse `Desktop Development with C++` in the `Desktop & Mobile` Section and then install. +1. During the installation, choose **Desktop Development with C++** in the **Desktop & Mobile** section. Then install the software -.. note:: + .. note:: - We recommend C++ compiler `Clang `_ and `Intel Compiler `_. + We recommend C++ compiler `Clang `_ and `Intel Compiler `_. -Install Miniforge -^^^^^^^^^^^^^^^^^ -Download and install `Miniforge `_. +1. Download and install `Miniforge3-Windows-x86_64.exe `__. Set Up Environment ^^^^^^^^^^^^^^^^^^ -#. Open a command line environment via cmd.exe. -#. Activate `MSVC` via below command:: +#. Open the command line environment via ``cmd.exe``. +#. Activate `MSVC` with the following command:: "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Auxiliary/Build/vcvars64.bat" -#. Activate `conda` via below command:: +#. Activate `conda` with the following command: + + .. code-block:: sh "C:/ProgramData/miniforge3/Scripts/activate.bat" -#. Create and activate customer conda environment:: +#. Create and activate a customer conda environment: + + .. code-block:: sh conda create -n inductor_cpu_windows python=3.10 -y conda activate inductor_cpu_windows + #. Install `PyTorch 2.5 `_ or later. -#. Use torchinductor on Windows CPU:: + +Using TorchInductor on Windows CPU +---------------------------------------- + +Here’s a simple example to demonstrate how to use TorchInductor: + +.. code-block:: python + import torch def foo(x, y): @@ -61,7 +77,9 @@ Set Up Environment opt_foo1 = torch.compile(foo) print(opt_foo1(torch.randn(10, 10), torch.randn(10, 10))) -#. Output of the above example:: +The code above returns the following output: + +.. code-block:: sh tensor([[-3.9074e-02, 1.3994e+00, 1.3894e+00, 3.2630e-01, 8.3060e-01, 1.1833e+00, 1.4016e+00, 7.1905e-01, 9.0637e-01, -1.3648e+00], @@ -87,4 +105,4 @@ Set Up Environment Conclusion ---------- -With this tutorial, we introduce how to use Inductor on Windows CPU with PyTorch 2.5 or later. +In this tutorial, we have learned how to use Inductor on Windows CPU with PyTorch 2.5 or later. From 7216e5c743c030e4c1a658049273018a5c91e679 Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Thu, 26 Sep 2024 09:14:42 -0700 Subject: [PATCH 08/17] Apply suggestions from code review --- prototype_source/inductor_windows_cpu.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 4d980eccab..206b20d0cd 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -24,7 +24,7 @@ This tutorial will guide you through the process of using TorchInductor on a Win * Miniforge for Windows Install the Required Software ---------------------- +------------------------------------------- First, let's install the required software. C++ compiler is required for TorchInductor optimization. We will use Microsoft Visual C++ (MSVC) for this example. @@ -40,8 +40,8 @@ We will use Microsoft Visual C++ (MSVC) for this example. 1. Download and install `Miniforge3-Windows-x86_64.exe `__. -Set Up Environment -^^^^^^^^^^^^^^^^^^ +Set Up the Environment +------------------------------ #. Open the command line environment via ``cmd.exe``. #. Activate `MSVC` with the following command:: From d2e1f27b379b3f32ceeebe65ec2543c786169e4a Mon Sep 17 00:00:00 2001 From: Svetlana Karslioglu Date: Thu, 26 Sep 2024 13:32:48 -0700 Subject: [PATCH 09/17] Apply suggestions from code review --- prototype_source/inductor_windows_cpu.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 206b20d0cd..157ab486f2 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -31,14 +31,14 @@ We will use Microsoft Visual C++ (MSVC) for this example. 1. Download and install `MSVC `_. -1. During the installation, choose **Desktop Development with C++** in the **Desktop & Mobile** section. Then install the software +2. During the installation, choose **Desktop Development with C++** in the **Desktop & Mobile** section. Then install the software - .. note:: +.. note:: We recommend C++ compiler `Clang `_ and `Intel Compiler `_. -1. Download and install `Miniforge3-Windows-x86_64.exe `__. +3. Download and install `Miniforge3-Windows-x86_64.exe `__. Set Up the Environment ------------------------------ @@ -54,10 +54,10 @@ Set Up the Environment "C:/ProgramData/miniforge3/Scripts/activate.bat" #. Create and activate a customer conda environment: - .. code-block:: sh + .. code-block:: sh - conda create -n inductor_cpu_windows python=3.10 -y - conda activate inductor_cpu_windows + conda create -n inductor_cpu_windows python=3.10 -y + conda activate inductor_cpu_windows #. Install `PyTorch 2.5 `_ or later. From 0bfbceb2c0b4f05eaaf314f5605548695e772124 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Fri, 27 Sep 2024 11:57:36 +0800 Subject: [PATCH 10/17] add alternative compiler based on svekars comments --- prototype_source/inductor_windows_cpu.rst | 33 +++++++++++++++++------ 1 file changed, 25 insertions(+), 8 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 157ab486f2..f6a0a75203 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -24,30 +24,30 @@ This tutorial will guide you through the process of using TorchInductor on a Win * Miniforge for Windows Install the Required Software -------------------------------------------- +----------------------------- First, let's install the required software. C++ compiler is required for TorchInductor optimization. We will use Microsoft Visual C++ (MSVC) for this example. 1. Download and install `MSVC `_. -2. During the installation, choose **Desktop Development with C++** in the **Desktop & Mobile** section. Then install the software +2. During the installation, choose **Desktop Development with C++** in the **Desktop & Mobile** section in **Workloads** table. Then install the software .. note:: We recommend C++ compiler `Clang `_ and `Intel Compiler `_. - + Please check `Alternative Compiler for better performance <#alternative-compiler-for-better-performance>`_. 3. Download and install `Miniforge3-Windows-x86_64.exe `__. Set Up the Environment ------------------------------- +---------------------- #. Open the command line environment via ``cmd.exe``. -#. Activate `MSVC` with the following command:: +#. Activate ``MSVC`` with the following command:: "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Auxiliary/Build/vcvars64.bat" -#. Activate `conda` with the following command: +#. Activate ``conda`` with the following command: .. code-block:: sh @@ -62,7 +62,7 @@ Set Up the Environment #. Install `PyTorch 2.5 `_ or later. Using TorchInductor on Windows CPU ----------------------------------------- +---------------------------------- Here’s a simple example to demonstrate how to use TorchInductor: @@ -102,7 +102,24 @@ The code above returns the following output: [ 5.7244e-04, 1.2799e+00, 1.3595e+00, 1.0907e+00, 3.7191e-01, 1.4062e+00, 1.3672e+00, 6.8502e-02, 8.5216e-01, 8.6046e-01]]) +Alternative Compiler for better performance +------------------------------------------- + +To enhance performance on Windows inductor, you can use the Intel Compiler or LLVM Compiler. However, they rely on the runtime libraries from Microsoft Visual C++ (MSVC). Therefore, your first step should be to install MSVC. + +Intel Compiler +^^^^^^^^^^^^^^ + +#. Download and install `Intel Compiler `_ with Windows version. +#. Set Windows Inductor Compiler via environment variable ``set CXX=icx-cl`` + +LLVM Compiler +^^^^^^^^^^^^^ + +#. Download and install `LLVM Compiler `_ and choose win64 version. +#. Set Windows Inductor Compiler via environment variable ``set CXX=clang-cl`` + Conclusion ---------- -In this tutorial, we have learned how to use Inductor on Windows CPU with PyTorch 2.5 or later. +With this tutorial, we introduce how to use Inductor on Windows CPU with PyTorch 2.5 or later. We can use Intel Compiler or LLVM Compiler to get better performance. From b6c69db12006fbae97a24b0dc391434eb4a10f4f Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Fri, 27 Sep 2024 12:00:23 +0800 Subject: [PATCH 11/17] add doc to prototype_index --- prototype_source/prototype_index.rst | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/prototype_source/prototype_index.rst b/prototype_source/prototype_index.rst index 1eaedb6a1d..c86ae85733 100644 --- a/prototype_source/prototype_index.rst +++ b/prototype_source/prototype_index.rst @@ -217,6 +217,13 @@ Prototype features are not available as part of binary distributions like PyPI o :link: ../prototype/inductor_cpp_wrapper_tutorial.html :tags: Model-Optimization +.. customcarditem:: + :header: Inductor Windows CPU Tutorial + :card_description: Speed up your models with Inductor On Windows CPU + :image: ../_static/img/thumbnails/cropped/generic-pytorch-logo.png + :link: ../prototype/inductor_windows_cpu.html + :tags: Model-Optimization + .. Distributed .. customcarditem:: :header: Flight Recorder Tutorial @@ -249,6 +256,7 @@ Prototype features are not available as part of binary distributions like PyPI o prototype/flight_recorder_tutorial.html prototype/graph_mode_dynamic_bert_tutorial.html prototype/inductor_cpp_wrapper_tutorial.html + prototype/inductor_windows_cpu.html prototype/pt2e_quantizer.html prototype/pt2e_quant_ptq.html prototype/pt2e_quant_qat.html From b6c5ea6f2f599719bcd9f62632fa546cc8ddc9f9 Mon Sep 17 00:00:00 2001 From: "Zheng, Zhaoqiong" Date: Fri, 27 Sep 2024 15:45:09 +0800 Subject: [PATCH 12/17] fix command format --- prototype_source/inductor_windows_cpu.rst | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index f6a0a75203..5febb5556e 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -44,7 +44,9 @@ Set Up the Environment ---------------------- #. Open the command line environment via ``cmd.exe``. -#. Activate ``MSVC`` with the following command:: +#. Activate ``MSVC`` with the following command: + + .. code-block:: sh "C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Auxiliary/Build/vcvars64.bat" #. Activate ``conda`` with the following command: @@ -54,10 +56,10 @@ Set Up the Environment "C:/ProgramData/miniforge3/Scripts/activate.bat" #. Create and activate a customer conda environment: - .. code-block:: sh + .. code-block:: sh - conda create -n inductor_cpu_windows python=3.10 -y - conda activate inductor_cpu_windows + conda create -n inductor_cpu_windows python=3.10 -y + conda activate inductor_cpu_windows #. Install `PyTorch 2.5 `_ or later. From 84a84f76d05c92a76c1aaa0f28c958ca484454a0 Mon Sep 17 00:00:00 2001 From: ZhaoqiongZ <106125927+ZhaoqiongZ@users.noreply.github.com> Date: Sun, 29 Sep 2024 17:55:32 +0800 Subject: [PATCH 13/17] Update prototype_source/inductor_windows_cpu.rst Co-authored-by: Svetlana Karslioglu --- prototype_source/inductor_windows_cpu.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 5febb5556e..c25aabbbea 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -104,7 +104,7 @@ The code above returns the following output: [ 5.7244e-04, 1.2799e+00, 1.3595e+00, 1.0907e+00, 3.7191e-01, 1.4062e+00, 1.3672e+00, 6.8502e-02, 8.5216e-01, 8.6046e-01]]) -Alternative Compiler for better performance +Using an Alternative Compiler for Better Performance ------------------------------------------- To enhance performance on Windows inductor, you can use the Intel Compiler or LLVM Compiler. However, they rely on the runtime libraries from Microsoft Visual C++ (MSVC). Therefore, your first step should be to install MSVC. From ae04689568e1936164ab743804f53312c7f12b5a Mon Sep 17 00:00:00 2001 From: ZhaoqiongZ <106125927+ZhaoqiongZ@users.noreply.github.com> Date: Sun, 29 Sep 2024 17:55:44 +0800 Subject: [PATCH 14/17] Update prototype_source/inductor_windows_cpu.rst Co-authored-by: Svetlana Karslioglu --- prototype_source/inductor_windows_cpu.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index c25aabbbea..7bf5d2537c 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -119,7 +119,7 @@ LLVM Compiler ^^^^^^^^^^^^^ #. Download and install `LLVM Compiler `_ and choose win64 version. -#. Set Windows Inductor Compiler via environment variable ``set CXX=clang-cl`` +#. Set Windows Inductor Compiler with the CXX environment variable ``set CXX=clang-cl``. Conclusion ---------- From c7ce4a0426c93e6a691e51e3487117f9dd6aed10 Mon Sep 17 00:00:00 2001 From: ZhaoqiongZ <106125927+ZhaoqiongZ@users.noreply.github.com> Date: Sun, 29 Sep 2024 17:55:53 +0800 Subject: [PATCH 15/17] Update prototype_source/inductor_windows_cpu.rst Co-authored-by: Svetlana Karslioglu --- prototype_source/inductor_windows_cpu.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index 7bf5d2537c..efcdea0e4c 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -124,4 +124,5 @@ LLVM Compiler Conclusion ---------- -With this tutorial, we introduce how to use Inductor on Windows CPU with PyTorch 2.5 or later. We can use Intel Compiler or LLVM Compiler to get better performance. +In this tutorial, we have learned how to use Inductor on Windows CPU with PyTorch. In addition, we discussed +further performance improvements with Intel Compiler and LLVM Compiler. From 51b154a48c344691aeab861bbd477b08a750fdb4 Mon Sep 17 00:00:00 2001 From: ZhaoqiongZ <106125927+ZhaoqiongZ@users.noreply.github.com> Date: Sun, 29 Sep 2024 17:56:03 +0800 Subject: [PATCH 16/17] Update prototype_source/inductor_windows_cpu.rst Co-authored-by: Svetlana Karslioglu --- prototype_source/inductor_windows_cpu.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index efcdea0e4c..ad839a8ad4 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -5,7 +5,7 @@ How to use TorchInductor on Windows CPU -TorchInductor is a new compiler backend that compiles FX Graphs generated by TorchDynamo into optimized C++/Triton kernels. +TorchInductor is a compiler backend that transforms FX Graphs generated by TorchDynamo into highly optimized C++/Triton kernels. This tutorial will guide you through the process of using TorchInductor on a Windows CPU. .. grid:: 2 From 8a8fe38ceef86d520798188a8a10cd8fe72effec Mon Sep 17 00:00:00 2001 From: ZhaoqiongZ <106125927+ZhaoqiongZ@users.noreply.github.com> Date: Sun, 29 Sep 2024 17:57:07 +0800 Subject: [PATCH 17/17] Update prototype_source/inductor_windows_cpu.rst Co-authored-by: Svetlana Karslioglu --- prototype_source/inductor_windows_cpu.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/prototype_source/inductor_windows_cpu.rst b/prototype_source/inductor_windows_cpu.rst index ad839a8ad4..eb10be5a3d 100644 --- a/prototype_source/inductor_windows_cpu.rst +++ b/prototype_source/inductor_windows_cpu.rst @@ -113,7 +113,7 @@ Intel Compiler ^^^^^^^^^^^^^^ #. Download and install `Intel Compiler `_ with Windows version. -#. Set Windows Inductor Compiler via environment variable ``set CXX=icx-cl`` +#. Set Windows Inductor Compiler with the CXX environment variable ``set CXX=icx-cl``. LLVM Compiler ^^^^^^^^^^^^^