Skip to content

Commit

Permalink
Update README (#398)
Browse files Browse the repository at this point in the history
* updated README

* updated

* put back some stuff

---------

Co-authored-by: Piyush Sharda <[email protected]>
  • Loading branch information
psharda and Piyush Sharda authored Sep 25, 2023
1 parent 033e4f3 commit a97b927
Showing 1 changed file with 2 additions and 4 deletions.
6 changes: 2 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,12 +104,10 @@ By default, Quokka compiles itself to run only on CPUs. Quokka can run on either
### NVIDIA GPUs
If you want to run on NVIDIA GPUs, re-build Quokka as shown below. (*CUDA >= 11.7 is required. Quokka is only supported on Volta V100 GPUs or newer models. Your MPI library **must** support CUDA-aware MPI.*)
```
cmake .. -DCMAKE_BUILD_TYPE=Release -DAMReX_GPU_BACKEND=CUDA -DAMREX_GPUS_PER_NODE=N -G Ninja
cmake .. -DCMAKE_BUILD_TYPE=Release -DAMReX_GPU_BACKEND=CUDA -DAMReX_SPACEDIM=3 -G Ninja
ninja -j6
```
where $N$ is the number of GPUs available per compute node.

**It is necessary to use `-DAMREX_GPUS_PER_NODE` to specify the number of GPUs per compute node. Without this, performance will be very poor. All GPUs on a node must be visible from each MPI rank on the node for efficient GPU-aware MPI communication to take place via CUDA IPC.** When using the SLURM job scheduler, this means that `--gpu-bind` should be set to `none`.
**All GPUs on a node must be visible from each MPI rank on the node for efficient GPU-aware MPI communication to take place via CUDA IPC.** When using the SLURM job scheduler, this means that `--gpu-bind` should be set to `none`.

The compiled test problems are in the test problem subdirectories in `build/src/`. Example scripts for running Quokka on compute clusters are in the `scripts/` subdirectory.

Expand Down

0 comments on commit a97b927

Please sign in to comment.