[OMPI users] Building Open MPI and default hostfile change does not go through

[Konstantinos Konstantinidis]

Hi,

I have built Open MPI 2.1.2 multiple times on Ubuntu 16.04 and then I add
the line

orte_default_hostfile=/etc/openmpi/openmpi-default-hostfile

to the file

/etc/openmpi/openmpi-mca-params.conf

and I execute

sudo chown myUsername /etc/openmpi/openmpi-default-hostfile

For some reason this change never goes through and each time I run a
program with mpirun only one local process runs. So I have to manually
specify my hostname with the --hostfile argument.

What can be the cause of this?

The exact series of commands I use for building is the following

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install g++
sudo apt-get install valgrind
sudo apt-get install libopenmpi-dev
sudo apt-get install gfortran
sudo apt-get install make
wget
https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz
tar -xvf openmpi-* && cd openmpi-*
./configure --prefix=/usr/local --enable-mpi-cxx --enable-debug
--enable-memchecker --with-valgrind=/usr
sudo make all install

Then, I add the following lines to the .bashrc file (Is this necessary?)

export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"

for setting the path and library path, respectively and of course reload
.bashrc.

Is the above way of installing Open MPI correct? I am really wondering
since I have no solid Linux knowledge.

[Konstantinos Konstantinidis]

Yeap, exactly the hostfile I have is of the form

node1 slots=1
node2 slots=1
node3 slots=1

where the above hostnames are resolved in ~/.ssh/config file which has
entries of the form

Host node1
HostName 192.168.0.100
User ubuntu
IdentityFile ~/.ssh/mykey.pem

and so on.

So the mpirun cannot pickup the hostfile by itself and I have to specify it
each time.

Can you provide some more detail? I'm not able to get this to fail (i.e.,
it seems to be working as expected for me).
For example, what's the contents of your /etc/openmpi/openmpi-default-hostfile
-- did you list some hostnames in there?

On May 11, 2018, at 4:43 AM, Konstantinos Konstantinidis <
Hi,
I have built Open MPI 2.1.2 multiple times on Ubuntu 16.04 and then I

add the line

orte_default_hostfile=/etc/openmpi/openmpi-default-hostfile
to the file
/etc/openmpi/openmpi-mca-params.conf
and I execute
sudo chown myUsername /etc/openmpi/openmpi-default-hostfile
For some reason this change never goes through and each time I run a

program with mpirun only one local process runs. So I have to manually
specify my hostname with the --hostfile argument.

What can be the cause of this?
The exact series of commands I use for building is the following
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install g++
sudo apt-get install valgrind
sudo apt-get install libopenmpi-dev
sudo apt-get install gfortran
sudo apt-get install make
wget https://www.open-mpi.org/software/ompi/v2.1/downloads/

openmpi-2.1.2.tar.gz

tar -xvf openmpi-* && cd openmpi-*
./configure --prefix=/usr/local --enable-mpi-cxx --enable-debug

--enable-memchecker --with-valgrind=/usr

sudo make all install
Then, I add the following lines to the .bashrc file (Is this necessary?)
export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
for setting the path and library path, respectively and of course reload

.bashrc.

Is the above way of installing Open MPI correct? I am really wondering

since I have no solid Linux knowledge.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

--
Jeff Squyres

[Gilles Gouaillardet]

Konstantinos,


Since you ran

configure --prefix=/usr/local

the system-wide config file should be in

/usr/local/etc/openmpi-default-hostfile


Note /usr/local is the default prefix, so you do not even need the
--prefix=/usr/local option


Cheers,


Gilles

Post by Konstantinos Konstantinidis
Yeap, exactly the hostfile I have is of the form
node1 slots=1
node2 slots=1
node3 slots=1
where the above hostnames are resolved in ~/.ssh/config file which has
entries of the form
Host node1
 HostName 192.168.0.100
 User ubuntu
 IdentityFile ~/.ssh/mykey.pem
and so on.
So the mpirun cannot pickup the hostfile by itself and I have to
specify it each time.
On Fri, May 11, 2018 at 4:02 PM, Jeff Squyres (jsquyres)
Can you provide some more detail?  I'm not able to get this to
fail (i.e., it seems to be working as expected for me).
For example, what's the contents of your
/etc/openmpi/openmpi-default-hostfile -- did you list some
hostnames in there?

On May 11, 2018, at 4:43 AM, Konstantinos Konstantinidis
Hi,
I have built Open MPI 2.1.2 multiple times on Ubuntu 16.04 and

then I add the line

orte_default_hostfile=/etc/openmpi/openmpi-default-hostfile
to the file
/etc/openmpi/openmpi-mca-params.conf
and I execute
sudo chown myUsername /etc/openmpi/openmpi-default-hostfile
For some reason this change never goes through and each time I

run a program with mpirun only one local process runs. So I have
to manually specify my hostname with the --hostfile argument.

What can be the cause of this?
The exact series of commands I use for building is the following
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install g++
sudo apt-get install valgrind
sudo apt-get install libopenmpi-dev
sudo apt-get install gfortran
sudo apt-get install make
wget

https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz
<https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz>

tar -xvf openmpi-* && cd openmpi-*
./configure --prefix=/usr/local --enable-mpi-cxx --enable-debug

--enable-memchecker --with-valgrind=/usr

sudo make all install
Then, I add the following lines to the .bashrc file (Is this

necessary?)

export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
for setting the path and library path, respectively and of

course reload .bashrc.

Is the above way of installing Open MPI correct? I am really

wondering since I have no solid Linux knowledge.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

<https://lists.open-mpi.org/mailman/listinfo/users>
--
Jeff Squyres
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Konstantinos Konstantinidis]

Thank you Gilles,

One more question. Do I need to add the following lines to the .bashrc file
after installing Open MPI?

export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"

Post by Gilles Gouaillardet
Konstantinos,
Since you ran
configure --prefix=/usr/local
the system-wide config file should be in
/usr/local/etc/openmpi-default-hostfile
Note /usr/local is the default prefix, so you do not even need the
--prefix=/usr/local option
Cheers,
Gilles

Post by Konstantinos Konstantinidis
Yeap, exactly the hostfile I have is of the form
node1 slots=1
node2 slots=1
node3 slots=1
where the above hostnames are resolved in ~/.ssh/config file which has
entries of the form
Host node1
HostName 192.168.0.100
User ubuntu
IdentityFile ~/.ssh/mykey.pem
and so on.
So the mpirun cannot pickup the hostfile by itself and I have to specify
it each time.
On Fri, May 11, 2018 at 4:02 PM, Jeff Squyres (jsquyres) <
Can you provide some more detail? I'm not able to get this to
fail (i.e., it seems to be working as expected for me).
For example, what's the contents of your
/etc/openmpi/openmpi-default-hostfile -- did you list some
hostnames in there?

On May 11, 2018, at 4:43 AM, Konstantinos Konstantinidis
Hi,
I have built Open MPI 2.1.2 multiple times on Ubuntu 16.04 and

then I add the line

orte_default_hostfile=/etc/openmpi/openmpi-default-hostfile
to the file
/etc/openmpi/openmpi-mca-params.conf
and I execute
sudo chown myUsername /etc/openmpi/openmpi-default-hostfile
For some reason this change never goes through and each time I

run a program with mpirun only one local process runs. So I have
to manually specify my hostname with the --hostfile argument.

What can be the cause of this?
The exact series of commands I use for building is the following
sudo apt-get update
sudo apt-get upgrade
sudo apt-get install g++
sudo apt-get install valgrind
sudo apt-get install libopenmpi-dev
sudo apt-get install gfortran
sudo apt-get install make
wget

https://www.open-mpi.org/software/ompi/v2.1/downloads/openmp
i-2.1.2.tar.gz
<https://www.open-mpi.org/software/ompi/v2.1/downloads/openm
pi-2.1.2.tar.gz>

tar -xvf openmpi-* && cd openmpi-*
./configure --prefix=/usr/local --enable-mpi-cxx --enable-debug

--enable-memchecker --with-valgrind=/usr

sudo make all install
Then, I add the following lines to the .bashrc file (Is this

necessary?)

export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
for setting the path and library path, respectively and of

course reload .bashrc.

Is the above way of installing Open MPI correct? I am really

wondering since I have no solid Linux knowledge.

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

<https://lists.open-mpi.org/mailman/listinfo/users>
-- Jeff Squyres
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Gilles Gouaillardet]

Konstantinos,


You need to double check that, your OS might have done it out of the box
for you already.


Once logged, you can

which mpirun

If it resolves to /usr/local/bin/mpirun, then there is no need to update
$PATH, and then

ldd /usr/local/bin/mpirun

If it correctly resolves to /usr/local/lib/libopen-pal.so and friends,
then there is no need

to update $LD_LIBRARY_PATH as well


Cheers,


Gilles

Post by Konstantinos Konstantinidis
Thank you Gilles,
One more question. Do I need to add the following lines to the .bashrc
file after installing Open MPI?
export PATH="$PATH:/usr/local/bin"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
On Sun, May 13, 2018 at 8:48 PM, Gilles Gouaillardet
Konstantinos,
Since you ran
configure --prefix=/usr/local
the system-wide config file should be in
/usr/local/etc/openmpi-default-hostfile
Note /usr/local is the default prefix, so you do not even need the
--prefix=/usr/local option
Cheers,
Gilles
Yeap, exactly the hostfile I have is of the form
node1 slots=1
node2 slots=1
node3 slots=1
where the above hostnames are resolved in ~/.ssh/config file
which has entries of the form
Host node1
 HostName 192.168.0.100
 User ubuntu
 IdentityFile ~/.ssh/mykey.pem
and so on.
So the mpirun cannot pickup the hostfile by itself and I have
to specify it each time.
On Fri, May 11, 2018 at 4:02 PM, Jeff Squyres (jsquyres)
    Can you provide some more detail?  I'm not able to get this to
    fail (i.e., it seems to be working as expected for me).
    For example, what's the contents of your
    /etc/openmpi/openmpi-default-hostfile -- did you list some
    hostnames in there?
    > On May 11, 2018, at 4:43 AM, Konstantinos Konstantinidis
    >
    > Hi,
    >
    > I have built Open MPI 2.1.2 multiple times on Ubuntu
16.04 and
    then I add the line
    >
    > orte_default_hostfile=/etc/openmpi/openmpi-default-hostfile
    >
    > to the file
    >
    > /etc/openmpi/openmpi-mca-params.conf
    >
    > and I execute
    >
    > sudo chown myUsername /etc/openmpi/openmpi-default-hostfile
    >
    > For some reason this change never goes through and each
time I
    run a program with mpirun only one local process runs. So
I have
    to manually specify my hostname with the --hostfile argument.
    >
    > What can be the cause of this?
    >
    > The exact series of commands I use for building is the
following
    >
    > sudo apt-get update
    > sudo apt-get upgrade
    > sudo apt-get install g++
    > sudo apt-get install valgrind
    > sudo apt-get install libopenmpi-dev
    > sudo apt-get install gfortran
    > sudo apt-get install make
    > wget
https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz
<https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz>
   
<https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz
<https://www.open-mpi.org/software/ompi/v2.1/downloads/openmpi-2.1.2.tar.gz>>
    > tar -xvf openmpi-* && cd openmpi-*
    > ./configure --prefix=/usr/local --enable-mpi-cxx
--enable-debug
    --enable-memchecker --with-valgrind=/usr
    > sudo make all install
    >
    > Then, I add the following lines to the .bashrc file (Is this
    necessary?)
    >
    > export PATH="$PATH:/usr/local/bin"
    > export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/lib"
    >
    > for setting the path and library path, respectively and of
    course reload .bashrc.
    >
    > Is the above way of installing Open MPI correct? I am really
    wondering since I have no solid Linux knowledge.
    > _______________________________________________
    > users mailing list
    > https://lists.open-mpi.org/mailman/listinfo/users
<https://lists.open-mpi.org/mailman/listinfo/users>
    <https://lists.open-mpi.org/mailman/listinfo/users
<https://lists.open-mpi.org/mailman/listinfo/users>>
    --     Jeff Squyres
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users
<https://lists.open-mpi.org/mailman/listinfo/users>
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users
<https://lists.open-mpi.org/mailman/listinfo/users>
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users