[OMPI users] Using OpenSHMEM with Shared Memory

[Benjamin Brock]

How can I run an OpenSHMEM program just using shared memory? I'd like to
use OpenMPI to run SHMEM programs locally on my laptop.

I understand that the old SHMEM component (Yoda?) was taken out, and that
UCX is now required. I have a build of OpenMPI with UCX as per the
directions on this random GitHub Page
<https://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX>
.

When I try to just `shmemrun`, I get a complaint about not haivng any splm
components available.

[***@shini kmer_hash]$ shmemrun -np 2 ./kmer_generic_hash
--------------------------------------------------------------------------
No available spml components were found!

This means that there are no components of this type installed on your
system or all the components reported that they could not be used.

This is a fatal error; your SHMEM process is likely to abort. Check the
output of the "ompi_info" command and ensure that components of this
type are available on your system. You may also wish to check the
value of the "component_path" MCA parameter and ensure that it has at
least one directory that contains valid MCA components.
--------------------------------------------------------------------------
[shini:16341] SPML ikrit cannot be selected
[shini:16342] SPML ikrit cannot be selected
[shini:16336] 1 more process has sent help message help-oshmem-memheap.txt
/ find-available:none-found
[shini:16336] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages


I tried fiddling with the MCA command-line settings, but didn't have any
luck. Is it possible to do this? Can anyone point me to some
documentation?

Thanks,

Ben

[Jeff Hammond]

Post by Benjamin Brock
How can I run an OpenSHMEM program just using shared memory? I'd like to
use OpenMPI to run SHMEM programs locally on my laptop.

It’s not Open-MPI itself but OSHMPI sits on top of any MPI-3 library and
has a mode to bypass MPI for one-sided if only used within a shared-memory
domain.


See https://github.com/jeffhammond/oshmpi and use --enable-smp-optimizations.
While I don’t actively maintain it and it doesn’t support the latest spec,
I’ll fix bugs and implement features on demand if users file GitHub issues.

Sorry for the shameless self-promotion but I know a few folks who use
OSHMPI specifically because of the SMP feature.

Sandia OpenSHMEM with OFI definitely works on shared-memory as well. I use
it for all of my Travis CI testing of SHMEM code on both Mac and Linux.

Jeff

Post by Benjamin Brock
I understand that the old SHMEM component (Yoda?) was taken out, and that
UCX is now required. I have a build of OpenMPI with UCX as per the
directions on this random GitHub Page
<https://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX>
.
When I try to just `shmemrun`, I get a complaint about not haivng any splm
components available.
--------------------------------------------------------------------------
No available spml components were found!
This means that there are no components of this type installed on your
system or all the components reported that they could not be used.
This is a fatal error; your SHMEM process is likely to abort. Check the
output of the "ompi_info" command and ensure that components of this
type are available on your system. You may also wish to check the
value of the "component_path" MCA parameter and ensure that it has at
least one directory that contains valid MCA components.
--------------------------------------------------------------------------
[shini:16341] SPML ikrit cannot be selected
[shini:16342] SPML ikrit cannot be selected
[shini:16336] 1 more process has sent help message help-oshmem-memheap.txt
/ find-available:none-found
[shini:16336] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
I tried fiddling with the MCA command-line settings, but didn't have any
luck. Is it possible to do this? Can anyone point me to some
documentation?
Thanks,
Ben
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Howard Pritchard]

HI Ben,

Could you set these environment variables and post the output ?

export OMPI_MCA_spml=ucx
export OMPI_MCA_spml_base_verbose=100

then run your test?

Also, what OS are you using?

Howard

Post by Jeff Hammond

Post by Benjamin Brock
How can I run an OpenSHMEM program just using shared memory? I'd like to
use OpenMPI to run SHMEM programs locally on my laptop.

It’s not Open-MPI itself but OSHMPI sits on top of any MPI-3 library and
has a mode to bypass MPI for one-sided if only used within a shared-memory
domain.
See https://github.com/jeffhammond/oshmpi and use --enable-smp-optimizations.
While I don’t actively maintain it and it doesn’t support the latest spec,
I’ll fix bugs and implement features on demand if users file GitHub issues.
Sorry for the shameless self-promotion but I know a few folks who use
OSHMPI specifically because of the SMP feature.
Sandia OpenSHMEM with OFI definitely works on shared-memory as well. I use
it for all of my Travis CI testing of SHMEM code on both Mac and Linux.
Jeff

Post by Benjamin Brock
I understand that the old SHMEM component (Yoda?) was taken out, and that
UCX is now required. I have a build of OpenMPI with UCX as per the
directions on this random GitHub Page
<https://github.com/openucx/ucx/wiki/OpenMPI-and-OpenSHMEM-installation-with-UCX>
.
When I try to just `shmemrun`, I get a complaint about not haivng any
splm components available.
------------------------------------------------------------
--------------
No available spml components were found!
This means that there are no components of this type installed on your
system or all the components reported that they could not be used.
This is a fatal error; your SHMEM process is likely to abort. Check the
output of the "ompi_info" command and ensure that components of this
type are available on your system. You may also wish to check the
value of the "component_path" MCA parameter and ensure that it has at
least one directory that contains valid MCA components.
------------------------------------------------------------
--------------
[shini:16341] SPML ikrit cannot be selected
[shini:16342] SPML ikrit cannot be selected
[shini:16336] 1 more process has sent help message
help-oshmem-memheap.txt / find-available:none-found
[shini:16336] Set MCA parameter "orte_base_help_aggregate" to 0 to see
all help / error messages
I tried fiddling with the MCA command-line settings, but didn't have any
luck. Is it possible to do this? Can anyone point me to some
documentation?
Thanks,
Ben
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

--
Jeff Hammond
http://jeffhammond.github.io/
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Benjamin Brock]

Here's what I get with those environment variables:

https://hastebin.com/ibimipuden.sql

I'm running Arch Linux (but with OpenMPI/UCX installed from source as
described in my earlier message).

Ben

[Howard Pritchard]

HI Ben,

I'm afraid this is bad news for using UCX. The problem is that when UCX
was configured/built, it did not
find a transport for doing one sided put/get transfers. If you're feeling
lucky, you may want to
install xpmem (https://github.com/hjelmn/xpmem) and rebuild UCX. This
requires building a device driver against
your kernel source and taking steps to getting the xpmem.ko loaded into the
kernel, etc.

There's an alternative however which works just fine on a laptop running
linux or osx. Check out

https://github.com/Sandia-OpenSHMEM/SOS/releases

and get the 1.4.0 release.

For build/install, follow the directions at

https://github.com/Sandia-OpenSHMEM/SOS/wiki/OFI-Build-Instructions

Note you will also need to install the MPICH hydra launcher as well.

Sandia OpenSHMEM over OFI libfabric uses TCP sockets as the fallback if
nothing else
is available. I use this version of OpenSHMEM if I'm doing SHMEM stuff on
my mac (no vm's).

Howard

Post by Benjamin Brock
https://hastebin.com/ibimipuden.sql
I'm running Arch Linux (but with OpenMPI/UCX installed from source as
described in my earlier message).
Ben