Building an AMPI+Kokkos application using CMake

[caschb]

Hello,

I'm trying to build the simplest MPI+Kokkos application with AMPI.

#include <Kokkos_Core.hpp>
#include <mpi.h>

int main(int argc, char **argv)
{
  MPI_Init(&argc, &argv);
  Kokkos::initialize(argc, argv);
  Kokkos::finalize();
  MPI_Finalize();
  return 0;
}

I'm also using a simple CMakeFile

cmake_minimum_required(VERSION 3.21)
project(main LANGUAGES CXX)

add_executable(main main.cpp)

find_package(MPI REQUIRED)
target_link_libraries(main MPI::MPI_CXX)

find_package(Kokkos)
target_link_libraries(main Kokkos::kokkos)

This setup works correctly when using the system MPI (MPICH), however I haven't been able to compile this program correctly when using AMPI.

I've tried to pass the AMPI compiler as an environment variable (export CC=ampicxx), as a CMake variable (-DCMAKE_CXX_COMPILER=ampicxx).

I've also tried deleting the MPI related lines from the CMakeLists in case they're not necessary for AMPI.

It's worth noting that I've compile both AMPI and Kokkos with CUDA support.

I'm wondering if I'm missing something or if I should modify my CMakeLists somehow?

Thanks in advance

[matthiasdiener]

I would try setting export MPICXX=/path/to/ampicxx. What is the error message you get when trying to compile?

[caschb]

Thank you for your prompt answer,
Depending on the steps I take I get different errors,
If I export MPICXX as you suggested AMPI doesn't get used, I can tell this is the case because the ampirun and charmrun scripts don't get generated and there's no reference to AMPI in the CMakeCache
EDIT: I also checked with ldd and it linked against the normal MPI and nothing related to AMPI or Charm++.

If I export CXX=/path/to/ampicxx instead, CMake runs without errors, but when I try to compile I get the following,

/home/casch/kokkos-ampi-test/main.cpp:1:10: fatal error: Kokkos_Core.hpp: No such file or directory
    1 | #include <Kokkos_Core.hpp>
      |          ^~~~~~~~~~~~~~~~~
compilation terminated.

This error doesn't happen when compiling with the normal MPI.

If instead I add -DMPI_CXX_COMPILER=/path/to/ampicxx to the CMake command, I get the following

-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
CMake Error at /opt/tools/cmake-3.27.5/share/cmake-3.27/Modules/FindPackageHandleStandardArgs.cmake:230 (message):
  Could NOT find MPI (missing: MPI_CXX_FOUND)