Skip to content

Commit

Permalink
update Setonix config (#676)
Browse files Browse the repository at this point in the history
### Description
This updates the Setonix profile to use the latest module versions that
are available. Pawsey updated the software stack in mid-June 2024 to
Cray PE 23.09.

I've compiled and tested the ShockCloud problem on Setonix GPUs. It
appears to work.

### Related issues
#569

### Checklist
_Before this pull request can be reviewed, all of these tasks should be
completed. Denote completed tasks with an `x` inside the square brackets
`[ ]` in the Markdown source below:_
- [x] I have added a description (see above).
- [x] I have added a link to any related issues see (see above).
- [x] I have read the [Contributing
Guide](https://github.com/quokka-astro/quokka/blob/development/CONTRIBUTING.md).
- [ ] I have added tests for any new physics that this PR adds to the
code.
- [ ] I have tested this PR on my local computer and all tests pass.
- [ ] I have manually triggered the GPU tests with the magic comment
`/azp run`.
- [x] I have requested a reviewer for this PR.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
  • Loading branch information
BenWibking and pre-commit-ci[bot] authored Jul 11, 2024
1 parent d719a59 commit 1731a92
Show file tree
Hide file tree
Showing 3 changed files with 8 additions and 31 deletions.
18 changes: 0 additions & 18 deletions scripts/setonix-cpu.profile

This file was deleted.

16 changes: 7 additions & 9 deletions scripts/setonix-gpu.profile
Original file line number Diff line number Diff line change
@@ -1,28 +1,26 @@
#!/bin/bash

source /opt/cray/pe/cpe/23.03/restore_lmod_system_defaults.sh

module load cmake/3.24.3
#source /opt/cray/pe/cpe/23.09/restore_lmod_system_defaults.sh
module load PrgEnv-cray
module load craype-accel-amd-gfx90a
module load rocm/5.2.3
module load cray-mpich
module load cce/15.0.1
module load cce/16.0.1
module load rocm/5.7.3

module load cray-hdf5
module load cray-python/3.9.13.1
module load cmake/3.27.7

# GPU-aware MPI
export MPICH_GPU_SUPPORT_ENABLED=1

# optimize ROCm/HIP compilation for MI250X
export AMREX_AMD_ARCH=gfx90a

# allow CMake to find Ascent
export Ascent_DIR=/software/projects/pawsey0807/bwibking/ascent_06082023/install/ascent-develop/lib/cmake/ascent/

# compiler environment hints
export CC=$(which cc)
export CXX=$(which CC)
export FC=$(which ftn)
export CFLAGS="-I${ROCM_PATH}/include"
export CXXFLAGS="-I${ROCM_PATH}/include"
export CXXFLAGS="-I${ROCM_PATH}/include -mllvm -amdgpu-function-calls=true"

5 changes: 1 addition & 4 deletions src/ShockCloud/cloud.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,7 @@ template <> struct quokka::EOS_Traits<ShockCloud> {
static constexpr double boltzmann_constant = C::k_B;
};

// global variables
namespace
{
/// global variables
// Problem properties (set inside problem_main())
bool sharp_cloud_edge = false; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
bool do_frame_shift = true; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
Expand All @@ -78,7 +76,6 @@ AMREX_GPU_MANAGED Real rho_wind = 0; // NOLINT(cppcoreguidelines-avoid-non-cons
AMREX_GPU_MANAGED Real v_wind = 0; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
AMREX_GPU_MANAGED Real P_wind = 0; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
AMREX_GPU_MANAGED Real delta_vx = 0; // NOLINT(cppcoreguidelines-avoid-non-const-global-variables)
} // namespace

template <> void RadhydroSimulation<ShockCloud>::setInitialConditionsOnGrid(quokka::grid grid)
{
Expand Down

0 comments on commit 1731a92

Please sign in to comment.