Environment

Create and activate a Python virtual environment

PyQUDA requires python>=3.7. You can check the version of your python3 by

python3 --version

We recommend using the virtual environment to install your Python packages. You can use venv, virtualenv or conda to manage your virtual environments. We recommend using the Python standard library venv instead of conda because of some OpenMPI issues.

Using venv

python3 -m venv .venv
source .venv/bin/activate

Using conda

conda create .venv
conda activate .venv

.venv can be any other name you like.

Install prerequisites

Cython, NumPy and mpi4py

pip install cython numpy mpi4py

CuPy

Install the prebuilt cupy package with your CUDA Toolkit version. You can check the version by

nvcc --version

And then install one of the packages below depending on the version

pip install cupy-cuda110
pip install cupy-cuda111
pip install cupy-cuda11x
pip install cupy-cuda12x

Check https://docs.cupy.dev/en/stable/install.html#installing-cupy for more information.

Install cupy from the source is possible but not recommended

pip install cupy

Modification with DTK

If you want to install cupy on DCU clusters, modify the cupy source code and then install it with DTK just like ROCm. Refer to https://docs.cupy.dev/en/stable/install.html#using-cupy-on-amd-gpu-experimental for detailed information.

Download the source code of CuPy version 12.3.0

wget https://github.com/cupy/cupy/releases/download/v12.3.0/cupy-12.3.0.tar.gz
tar -xzvf cupy-12.3.0.tar.gz
cd cupy-12.3.0

Apply the patch

diff --git a/install/cupy_builder/_features.py b/install/cupy_builder/_features.py
index d12de78c3..8c9ac830a 100644
--- a/install/cupy_builder/_features.py
+++ b/install/cupy_builder/_features.py
@@ -173,7 +173,7 @@ def get_features(ctx: Context) -> Dict[str, Feature]:
             'hiprand',
             'hipsparse',
             'rocfft',
-            'roctx64',
+            #'roctx64',
             'rocblas',
             'rocsolver',
             'rocsparse',

Build and install CuPy from source

export CUPY_INSTALL_USE_HIP=1
export ROCM_HOME=/path/to/dtk-23.10
pip install .

PyTorch (Optional)

torch is an alternative to cupy.

python3 -m pip install torch