MPI shared memory

[hmenke]

All tests were run with multiple nodes and different number of slots on each node:

$ cat hostfile 
pcscqm04 slots=4
pcscqm05 slots=3
pcscqm06 slots=5
pcscqm07 slots=2
$ mpirun -hostfile ./hostfile build/test/c++/mpi_window

Some ideas:

• MPI allocator

  Similar to std::allocator implement a shared_allocator and a distributed_shared_allocator, such that one can use e.g. std::vector<double, mpi::shared_allocator<double>>

  Questions:

  • Should this be in mpi or in nda?
  • For shared memory one can probably use a best guess and use split_shared() on the default communicator, but for distributed shared memory there needs to be internode communication and that is not easily guessed from the default communicator. Maybe a global hash table with allocation information is needed?
  • For shared memory race conditions must be prevented somehow.
  • On top of that, for distributed shared memory access must be fenced and broadcasted between nodes. That's not so easy to abstract away.

[hmenke]

Can we make this non-MPI compatible?