[OMPI users] Setting LD_LIBRARY

Discussion:

[OMPI users] Setting LD_LIBRARY_PATH for orted

Jackson, Gary L.

2017-08-21 16:06:34 UTC

Iâm using a binary distribution of OpenMPI 1.10.2. As linked, it requires certain shared libraries outside of OpenMPI for orted itself to start. So, passing in LD_LIBRARY_PATH with the â-xâ flag to mpirun doesnât do anything:

$ mpirun âhostfile ${HOSTFILE} -N 1 -n 2 -x LD_LIBRARY_PATH hostname
/path/to/orted: error while loading shared libraries: LIBRARY.so: cannot open shared object file: No such file or directory
--------------------------------------------------------------------------
ORTE was unable to reliably start one or more daemons.
This usually is caused by:

* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-orterun-prefix-by-default

* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.

* the inability to write startup files into /tmp (--tmpdir/orte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.

* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.

* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
--------------------------------------------------------------------------

How do I get around this cleanly? This works just fine when I set LD_LIBRARY_PATH in my .bashrc, but Iâd rather not pollute that if I can avoid it.

--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory

Reuti

2017-08-21 16:15:40 UTC

Permalink

Hi,

$ mpirun –hostfile ${HOSTFILE} -N 1 -n 2 -x LD_LIBRARY_PATH hostname
/path/to/orted: error while loading shared libraries: LIBRARY.so: cannot open shared object file: No such file or directory
--------------------------------------------------------------------------
ORTE was unable to reliably start one or more daemons.
* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-orterun-prefix-by-default
* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.
* the inability to write startup files into /tmp (--tmpdir/orte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.
* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.
* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
--------------------------------------------------------------------------
How do I get around this cleanly? This works just fine when I set LD_LIBRARY_PATH in my .bashrc, but I’d rather not pollute that if I can avoid it.

Do you set or extend the LD_LIBRARY_PATH in your .bashrc?

-- Reuti

Gilles Gouaillardet

2017-08-22 00:04:11 UTC

Permalink

Gary,

one option (as mentioned in the error message) is to configure Open MPI
with --enable-orterun-prefix-by-default.

this will force the build process to use rpath, so you do not have to
set LD_LIBRARY_PATH

this is the easiest option, but cannot be used if you plan to relocate
the Open MPI installation directory.

an other option is to use a wrapper for orted.

mpirun --mca orte_launch_agent /.../myorted ...

where myorted is a script that looks like

#!/bin/sh

export LD_LIBRARY_PATH=...

exec /.../bin/orted "$@"

you can make this setting system-wide by adding the following line to
/.../etc/openmpi-mca-params.conf

orte_launch_agent = /.../myorted

Cheers,

Gilles

I’m using a binary distribution of OpenMPI 1.10.2. As linked, it
requires certain shared libraries outside of OpenMPI for orted itself
to start. So, passing in LD_LIBRARY_PATH with the “-x” flag to mpirun
$ mpirun –hostfile ${HOSTFILE} -N 1 -n 2 -x LD_LIBRARY_PATH hostname
cannot open shared object file: No such file or directory
--------------------------------------------------------------------------
ORTE was unable to reliably start one or more daemons.
* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-orterun-prefix-by-default
* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.
* the inability to write startup files into /tmp
(--tmpdir/orte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.
* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.
* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
--------------------------------------------------------------------------
How do I get around this cleanly? This works just fine when I set
LD_LIBRARY_PATH in my .bashrc, but I’d rather not pollute that if I
can avoid it.
--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

John Hearns via users

2017-08-22 07:50:13 UTC

Permalink

Gary, are you using Modules?
http://www.admin-magazine.com/HPC/Articles/Environment-Modules

Post by Gilles Gouaillardet
Gary,
one option (as mentioned in the error message) is to configure Open MPI
with --enable-orterun-prefix-by-default.
this will force the build process to use rpath, so you do not have to set
LD_LIBRARY_PATH
this is the easiest option, but cannot be used if you plan to relocate the
Open MPI installation directory.
an other option is to use a wrapper for orted.
mpirun --mca orte_launch_agent /.../myorted ...
where myorted is a script that looks like
#!/bin/sh
export LD_LIBRARY_PATH=...
you can make this setting system-wide by adding the following line to
/.../etc/openmpi-mca-params.conf
orte_launch_agent = /.../myorted
Cheers,
Gilles

Post by Jackson, Gary L.
Iâm using a binary distribution of OpenMPI 1.10.2. As linked, it requires
certain shared libraries outside of OpenMPI for orted itself to start. So,
$ mpirun âhostfile ${HOSTFILE} -N 1 -n 2 -x LD_LIBRARY_PATH hostname
/path/to/orted: error while loading shared libraries: LIBRARY.so: cannot
open shared object file: No such file or directory
------------------------------------------------------------
--------------
ORTE was unable to reliably start one or more daemons.
* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-orterun-prefix-by-default
* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.
* the inability to write startup files into /tmp
(--tmpdir/orte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.
* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.
* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
------------------------------------------------------------
--------------
How do I get around this cleanly? This works just fine when I set
LD_LIBRARY_PATH in my .bashrc, but Iâd rather not pollute that if I can
avoid it.
--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

Gilles Gouaillardet

2017-08-22 13:34:51 UTC

Permalink

John,

Modules (both legacy and LMod) are great but might not be relevant here :
my understanding is that in this case, "module load openmpi" would be
required in the .bashrc,
and this is something that should be avoided.

Cheers,

Gilles

On Tue, Aug 22, 2017 at 4:50 PM, John Hearns via users

Post by John Hearns via users
Gary, are you using Modules?
http://www.admin-magazine.com/HPC/Articles/Environment-Modules

I’m using a binary distribution of OpenMPI 1.10.2. As linked, it requires
certain shared libraries outside of OpenMPI for orted itself to start. So,
$ mpirun –hostfile ${HOSTFILE} -N 1 -n 2 -x LD_LIBRARY_PATH hostname
/path/to/orted: error while loading shared libraries: LIBRARY.so: cannot
open shared object file: No such file or directory
--------------------------------------------------------------------------
ORTE was unable to reliably start one or more daemons.
* not finding the required libraries and/or binaries on
one or more nodes. Please check your PATH and LD_LIBRARY_PATH
settings, or configure OMPI with --enable-orterun-prefix-by-default
* lack of authority to execute on one or more specified nodes.
Please verify your allocation and authorities.
* the inability to write startup files into /tmp
(--tmpdir/orte_tmpdir_base).
Please check with your sys admin to determine the correct location to use.
* compilation of the orted with dynamic libraries when static are required
(e.g., on Cray). Please check your configure cmd line and consider using
one of the contrib/platform definitions for your system type.
* an inability to create a connection back to mpirun due to a
lack of common network interfaces and/or no route found between
them. Please check network connectivity (including firewalls
and network routing requirements).
--------------------------------------------------------------------------
How do I get around this cleanly? This works just fine when I set
LD_LIBRARY_PATH in my .bashrc, but I’d rather not pollute that if I can
avoid it.
--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

Jackson, Gary L.

2017-08-22 15:55:46 UTC

Permalink

I’m using a build of OpenMPI provided by a third party.

--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory

On 8/21/17, 8:04 PM, "users on behalf of Gilles Gouaillardet" <users-***@lists.open-mpi.org on behalf of ***@rist.or.jp> wrote:

Gary,

one option (as mentioned in the error message) is to configure Open MPI
with --enable-orterun-prefix-by-default.

this will force the build process to use rpath, so you do not have to
set LD_LIBRARY_PATH

this is the easiest option, but cannot be used if you plan to relocate
the Open MPI installation directory.

an other option is to use a wrapper for orted.

mpirun --mca orte_launch_agent /.../myorted ...

where myorted is a script that looks like

#!/bin/sh

export LD_LIBRARY_PATH=...

exec /.../bin/orted "$@"

you can make this setting system-wide by adding the following line to
/.../etc/openmpi-mca-params.conf

orte_launch_agent = /.../myorted

Cheers,

Gilles

_______________________________________________
users mailing list
***@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Bennet Fauber

2017-08-22 16:10:02 UTC

Permalink

Would

$ mpirun -x LD_LIBRARY_PATH ...

work here? I think from the man page for mpirun that should request
that it would would export the currently set value of LD_LIBRARY_PATH
to the remote nodes prior to executing the command there.

-- bennet

On Tue, Aug 22, 2017 at 11:55 AM, Jackson, Gary L.

Post by Jackson, Gary L.
I’m using a build of OpenMPI provided by a third party.
--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory
Gary,
one option (as mentioned in the error message) is to configure Open MPI
with --enable-orterun-prefix-by-default.
this will force the build process to use rpath, so you do not have to
set LD_LIBRARY_PATH
this is the easiest option, but cannot be used if you plan to relocate
the Open MPI installation directory.
an other option is to use a wrapper for orted.
mpirun --mca orte_launch_agent /.../myorted ...
where myorted is a script that looks like
#!/bin/sh
export LD_LIBRARY_PATH=...
you can make this setting system-wide by adding the following line to
/.../etc/openmpi-mca-params.conf
orte_launch_agent = /.../myorted
Cheers,
Gilles

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

r***@open-mpi.org

2017-08-22 17:07:06 UTC

Permalink

I’m afraid not - that only applies the variable to the application, not the daemons.

Truly, your only real option is to put something in your .bashrc since you cannot modify the configure.

Or, if you are running in a managed environment, you can ask to have your resource manager forward your environment to the allocated nodes.

Post by Bennet Fauber
Would
$ mpirun -x LD_LIBRARY_PATH ...
work here? I think from the man page for mpirun that should request
that it would would export the currently set value of LD_LIBRARY_PATH
to the remote nodes prior to executing the command there.
-- bennet
On Tue, Aug 22, 2017 at 11:55 AM, Jackson, Gary L.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

Jackson, Gary L.

2017-08-22 17:43:05 UTC

Permalink

Yup. It looks like I’m stuck with .bashrc.

Thank you all for the suggestions.

--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory

On 8/22/17, 1:07 PM, "users on behalf of ***@open-mpi.org" <users-***@lists.open-mpi.org on behalf of ***@open-mpi.org> wrote:

I’m afraid not - that only applies the variable to the application, not the daemons.

Truly, your only real option is to put something in your .bashrc since you cannot modify the configure.

Or, if you are running in a managed environment, you can ask to have your resource manager forward your environment to the allocated nodes.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
***@lists.open-mpi.org
https://lists.open-mpi.org/mailman/listinfo/users

Gilles Gouaillardet

2017-08-22 21:56:56 UTC

Permalink

Not really,

you have the option of using an orte_launch_agent i described in a
previous email

Cheers,

Gilles

On Wed, Aug 23, 2017 at 2:43 AM, Jackson, Gary L.

Post by Jackson, Gary L.
Yup. It looks like I’m stuck with .bashrc.
Thank you all for the suggestions.
--
Gary Jackson, Ph.D.
Johns Hopkins University Applied Physics Laboratory
I’m afraid not - that only applies the variable to the application, not the daemons.
Truly, your only real option is to put something in your .bashrc since you cannot modify the configure.
Or, if you are running in a managed environment, you can ask to have your resource manager forward your environment to the allocated nodes.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

Reuti

2017-08-22 20:34:02 UTC

Permalink

There is the tool `chrpath` to change rpath and runpath inside a binary/library. This has to match relocated directory then.

-- Reuti

an other option is to use a wrapper for orted.
mpirun --mca orte_launch_agent /.../myorted ...
where myorted is a script that looks like
#!/bin/sh
export LD_LIBRARY_PATH=...
you can make this setting system-wide by adding the following line to /.../etc/openmpi-mca-params.conf
orte_launch_agent = /.../myorted
Cheers,
Gilles

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users