[OMPI users] MPI flavor-agnostic libraries

[Gregory D Abram]

I'd like to release binary libraries that use MPI but are agnostic as to
which flavor or MPI (eg. OpenMPI, MVAPICH, LAM...) that is used by the
application linking the libraries. I've seen that there are some
significant differences that stand in the way, for example that MPI_Comm is
a pointer on OpenMPI and an integer on MVAPICH. I can see some ways that
might work, but they are pretty complex - for example, I could create an
intercept library that loads a real MPI library explicitly and do whatever
needs be done (for example, translating MPI_Comm parameters). Does anyone
know of anything that might help?

[Jeff Squyres]

Post by Gregory D Abram
I'd like to release binary libraries that use MPI but are agnostic
as to which flavor or MPI (eg. OpenMPI, MVAPICH, LAM...) that is
used by the application linking the libraries. I've seen that there
are some significant differences that stand in the way, for example
that MPI_Comm is a pointer on OpenMPI and an integer on MVAPICH. I
can see some ways that might work, but they are pretty complex - for
example, I could create an intercept library that loads a real MPI
library explicitly and do whatever needs be done (for example,
translating MPI_Comm parameters). Does anyone know of anything that
might help?

This is the age-old "there is no MPI ABI" issue. :-\ The root of the
issue is that MPI is [at least currently] defined to be source code
compatible, not binary compatible.

A "MorphMPI" solution has been proposed and discussed many times
(similar to what you describe -- a thin intercept layer than acts as a
translator to multiple back-end MPI implementations, probably loaded
dynamically at run-time), but no one has implemented a general purpose
solution that could be standardized across/used by any MPI software
package.

Most MPI software packages either distribute binaries for each
different MPI implementation that they want to support or distribute
source code and allow themselves to be compiled against multiple
different MPI implementations (which, these days, isn't too
difficult). Neither of these are optimal solutions, obviously.

[Mattijs Janssens]

We did something similar in OpenFOAM by encapsulating our communications
routines into a separate shared library which gets loaded through
LD_LIBRARY_PATH.

The api of this library does not expose any mpi specifics so for a new MPI
version we only have to recompile our small communications library against
the new MPI and the rest of the code works as normal.

Then we can just use LD_LIBRARY_PATH to pick which communications library to
load and hence which mpi to use.

Post by Gregory D Abram
I'd like to release binary libraries that use MPI but are agnostic as to
which flavor or MPI (eg. OpenMPI, MVAPICH, LAM...) that is used by the
application linking the libraries. I've seen that there are some
significant differences that stand in the way, for example that MPI_Comm is
a pointer on OpenMPI and an integer on MVAPICH. I can see some ways that
might work, but they are pretty complex - for example, I could create an
intercept library that loads a real MPI library explicitly and do whatever
needs be done (for example, translating MPI_Comm parameters). Does anyone
know of anything that might help?

--
Mattijs Janssens

OpenCFD Ltd.
9 Albert Road,
Caversham,
Reading RG4 7AN.
Tel: [phone]
Email: ***@OpenCFD.co.uk
URL: http://www.OpenCFD.co.uk