PackagingDrafts/MPI

This is a draft document

Introduction

Message Passing Interface (MPI) is an API for parallelization of programs across multiple nodes and has been around since 1994 [1]. MPI can also be used for parallelization on SMP machines and is considered very efficient in it too (close to 100% scaling on parallelizable code as compared to ~80% commonly obtained with threads due to unoptimal memory allocation on NUMA machines). Before MPI, about every manufacturer of supercomputers had their own programming language for writing programs; MPI made porting software easy.

There are many MPI implementations available, such as LAM-MPI (obsoleted by Open MPI), Open MPI (the MPI compiler used in RHEL), MPICH (Not yet in Fedora), MPICH2 and MVAPICH1 and MVAPICH2 (Not yet in Fedora).

As some MPI libraries work better on some hardware than others, and some software works best with some MPI library, the selection of the library used must be done on a user-level basis. Also, people doing high performance computing may want to use more efficient compilers, so one must be able to have many versions compiled with different compilers of the same library installed at the same time. This must be taken into account when writing spec files.

Packaging of MPI compilers

MPI compilers MUST be installed (including binaries, man pages, etc) in %{_libdir}/%{name}/%{version}-<compiler>, where <compiler> is normally gcc in Fedora.

The runtime of MPI compilers (mpirun, the libraries, the manuals etc) MUST be packaged into %{name}, and the development headers and libraries into %{name}-devel.

As the compiler is installed outside PATH (for a valid reason), one needs to load the relevant variables before being able to use the compiler or run MPI programs. This is done using environment modules.

The module file MUST prepend the MPI bindir {_libdir}/%{name}/%{version}-<compiler>/bin into the users PATH and set LD_LIBRARY_PATH to {_libdir}/%{name}/%{version}-<compiler>/lib. The module MUST provide environment variables $MPI_HOME, $MPI_BIN and $MPI_LIB, which should point to %{_libdir}/%{name}/%{version}-<compiler>, %{_libdir}/%{name}/%{version}-<compiler>/bin and %{_libdir}/%{name}/%{version}-<compiler>/lib, respectively.

MUST: By default, no files are placed in /etc/ld.so.conf.d. If the packager wishes to provide alternatives support, it MUST be placed in a subpackage along with the ld.so.conf.d file so that alternatives support does not need to be installed if not wished for.

The MPI compiler package MUST provide an RPM macro that makes loading and unloading the support easy in spec files, e.g. by placing the following in /etc/rpm/macros.openmpi

%_openmpi_load \
 . /etc/profile.d/modules.sh; \
 module load openmpi-%{_arch}; \
 export CFLAGS="$CFLAGS %{optflags}"
%_openmpi_unload \
 . /etc/profile.d/modules.sh; \
 module unload openmpi-%{_arch};

loading and unloading the compiler in spec files is as easy as %{_openmpi_load} and %{_openmpi_unload}.

Note about compiler flags
If the environment module sets compiler flags such as CFLAGS, the RPM macro MUST make the flags use the Fedora optimization flags %{optflags}.

Packaging of MPI software

Software that supports MPI MUST be packaged also in serial mode [i.e. no MPI], if it is supported by upstream. (for instance: foo).

The packager MUST package at least a version compiled against Open MPI. Packages made against other MPI compilers in Fedora SHOULD be made, but that is left up to the maintainer. The MPI enabled bits MUST be placed in a subpackage with the suffix denoting the MPI compiler used (for instance: foo-mpi for Open MPI [the traditional MPI compiler in Fedora] or foo-mpich2 for MPICH2).

To prevent name clashes, there are two possibilities in the installation location:

1. Placing in system directories
   • The binaries of the software placed in %{_bindir} MUST be suffixed with the name of the MPI compiler (e.g. bar_mpi [for Open MPI] or bar_mpich2 [for MPICH2]).
   • The libraries of the software placed in %{_libdir} MUST be suffixed with the name of the MPI compiler (e.g. libbar_mpi.so [for Open MPI] or libbar_mpich2.so [for MPICH2]).
2. Placing in a separate directory
   • The software MUST be installed in %{_libdir}/%{name}/%{version}-%{_mpi_compiler}/ (e.g. code>%{_libdir}/foo/1.0-openmpi-gcc/, including libraries and man files.
   • An environment module enabling the use of the software MUST be written and be made available as /etc/modulefiles/%{name}-%{compiler}-%{_arch}. The module MUST require the module of the used compiler. More info on environment modules.

The packages MUST have explicit requires on the used MPI runtime, as rpm might not pick up the correct version. - needs to be checked, at least libmpi is provided by all of them(?)