Process migration for OpenMPI

OpenMPI is a popular implementation of the MPI standard that is relatively easy to use and extend. For these reason we have chosen OpenMPI for developing support for migratable MPI processes using the MOSIX framework. Our code was submitted to the OpenMPI community for their approval to commit the patch into the mainline.

For now, the patches for both the stable 1.6 branch and the development trunk are available. These patches are released under the OpenMPI community license http://www.open-mpi.org/community/license.php

Please contact us if you have any questions or suggestions.

Installation

1. Download the developers version of the OMPI source code from the OMPI repositories, see instructions at http://www.open-mpi.org/svn/ . Make sure the file "autogen.sh" is located in the head folder. For example, run:

svn co http://svn.open-mpi.org/svn/ompi/trunk

2. Download the patch file according to the OpenMPI version you have chosen:

OpenMPI 1.6.x http://www.MOSIX.org/mpi/patch-openmpi-branch
OpenMPI trunk http://www.MOSIX.org/mpi/patch-openmpi-trunk-no-libevent

3. Enter the topmost folder of the OMPI you've just downloaded and apply the patch, by running:

cd openmpi
patch -p0 < mosix_ompi_trunk.patch

4. Build the OMPI source into binary. This step is where you can add or remove additional features. Make sure you do not skip the "autogen" phase, otherwise the patch will not be detected. MOSIX support will be included by default.

For example, you can run:

./autogen.sh
./configure
make
make install

Running OpenMPI over MOSIX

After the OpenMPI is built successfully using the patch described above, running jobs with "mpirun" should automatically use the MOSIX support to launch migratable processes. In order to fine-tune the launched application, or use advanced MOSIX features such as multi-cluster, flags can be passed on to "mpirun" like any other flags.

The MOSIX support patch for OMPI consists of 3 components, each with its own flags:

1. BTL - Responsible for send/recv actions.

2. ODLS - Responsible for launching the application (typically uses fork syscall).

3. RAS - Responsible for the job resource allocation.

In order to get the list of available flags, run:

ompi_info --param <component> mosix

For example, for launching an OpenMPI job with all processes locked (prevent migration) run:

mpirun -mca odls_mosix_lock=1 executable