Allan Overstreet
2017-05-26 22:28:24 UTC
I have been having some issues with using openmpi with tcp over IPoIB
and openib. The problems arise when I run a program that uses basic
collective communication. The two programs that I have been using are
attached.
*** IPoIB ***
The mpirun command I am using to run mpi over IPoIB is,
mpirun --mca oob_tcp_if_include 192.168.1.0/24 --mca btl_tcp_include
10.1.0.0/24 --mca pml ob1 --mca btl tcp,sm,vader,self -hostfile nodes
-np 8 ./avg 8000
This program will appear to run on the nodes, but will sit at 100% CPU
and use no memory. On the host node an error will be printed,
[sm1][[58411,1],0][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
Using another program,
mpirun --mca oob_tcp_if_include 192.168.1.0/24 --mca btl_tcp_if_include
10.1.0.0/24 --mca pml ob1 --mca btl tcp,sm,vader,self -hostfile nodes
-np 8 ./congrad 800
Produces the following result. This program will also run on the nodes
sm1, sm2, sm3, and sm4 at 100% and use no memory.
[sm3][[61383,1],4][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.5 failed: No route to host (113)
[sm4][[61383,1],6][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.4 failed: No route to host (113)
[sm2][[61383,1],3][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.2 failed: No route to host (113)
[sm3][[61383,1],5][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.5 failed: No route to host (113)
[sm4][[61383,1],7][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.4 failed: No route to host (113)
[sm2][[61383,1],2][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.2 failed: No route to host (113)
[sm1][[61383,1],0][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
[sm1][[61383,1],1][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
*** openib ***
Running the congrad program over openib will produce the result,
mpirun --mca btl self,sm,openib --mca mtl ^psm --mca btl_tcp_if_include
10.1.0.0/24 -hostfile nodes -np 8 ./avg 800
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
--------------------------------------------------------------------------
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
Open MPI stopped checking at the first component that it did not find.
Host: sm2.overst.local
Framework: btl
Component: openib
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
mca_bml_base_open() failed
--> Returned "Not found" (-13) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[sm1.overst.local:32239] [[57506,0],1] usock_peer_send_blocking: send()
to socket 29 failed: Broken pipe (32)
[sm1.overst.local:32239] [[57506,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32239] [[57506,0],1]-[[57506,1],1] usock_peer_accept:
usock_peer_send_connect_ack failed
[sm1.overst.local:32239] [[57506,0],1] usock_peer_send_blocking: send()
to socket 27 failed: Broken pipe (32)
[sm1.overst.local:32239] [[57506,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32239] [[57506,0],1]-[[57506,1],0] usock_peer_accept:
usock_peer_send_connect_ack failed
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[smd:31760] 4 more processes have sent help message help-mca-base.txt /
find-available:not-valid
[smd:31760] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
[smd:31760] 4 more processes have sent help message help-mpi-runtime.txt
/ mpi_init:startup:internal-failure
=== Later errors printed out on the host node ===
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm3
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm1
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm2
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm4
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
The ./avg process was not created on any of the nodes.
Running the ./congrad program,
mpirun --mca btl self,sm,openib --mca mtl ^psm --mca btl_tcp_if_include
10.1.0.0/24 -hostfile nodes -np 8 ./congrad 800
Will results in the following errors,
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
--------------------------------------------------------------------------
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
Open MPI stopped checking at the first component that it did not find.
Host: sm3.overst.local
Framework: btl
Component: openib
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
mca_bml_base_open() failed
--> Returned "Not found" (-13) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[sm1.overst.local:32271] [[57834,0],1] usock_peer_send_blocking: send()
to socket 29 failed: Broken pipe (32)
[sm1.overst.local:32271] [[57834,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32271] [[57834,0],1]-[[57834,1],0] usock_peer_accept:
usock_peer_send_connect_ack failed
[sm1.overst.local:32271] [[57834,0],1] usock_peer_send_blocking: send()
to socket 27 failed: Broken pipe (32)
[sm1.overst.local:32271] [[57834,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32271] [[57834,0],1]-[[57834,1],1] usock_peer_accept:
usock_peer_send_connect_ack failed
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[smd:32088] 5 more processes have sent help message help-mca-base.txt /
find-available:not-valid
[smd:32088] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
[smd:32088] 5 more processes have sent help message help-mpi-runtime.txt
/ mpi_init:startup:internal-failure
Using these mpirun commands will successfully run using a test program
that is only using point to point communication.
The nodes are interconnected in the following way.
__ _
[__]|=|
/::/|_|
HOST: smd
Dual 1Gb Ethernet Bonded
.-------------> Bond0 IP: 192.168.1.200
| Infiniband Card: MHQH29B-XTR <------------.
| Ib0 IP: 10.1.0.1 |
| OS: Ubuntu Mate |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: sm1 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.196 |
| Infiniband Card: QLOGIC QLE7340 <---------|
| Ib0 IP: 10.1.0.2 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
|---------. /::/|_| |
| | HOST: sm2 |
| | Dual 1Gb Ethernet Bonded |
| '---> Bond0 IP: 192.168.1.199 |
__________ Infiniband Card: QLOGIC QLE7340 __________
[_|||||||_°] Ib0 IP: 10.1.0.3 [_|||||||_°]
[_|||||||_°] OS: Centos 7 Minimal [_|||||||_°]
[_|||||||_°] __ _ [_|||||||_°]
Gb Ethernet Switch [__]|=| Voltaire 4036
QDR Switch
| /::/|_| |
| HOST: sm3 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.203 |
| Infiniband Card: QLOGIC QLE7340 <----------|
| Ib0 IP: 10.1.0.4 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: sm4 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.204 |
| Infiniband Card: QLOGIC QLE7340 <----------|
| Ib0 IP: 10.1.0.5 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: dl580 |
| Dual 1Gb Ethernet Bonded |
'-------------> Bond0 IP: 192.168.1.201 |
Infiniband Card: QLOGIC QLE7340 <----------'
Ib0 IP: 10.1.0.6
OS: Centos 7 Minimal
Thanks for the help again.
Sincerely,
Allan Overstreet
and openib. The problems arise when I run a program that uses basic
collective communication. The two programs that I have been using are
attached.
*** IPoIB ***
The mpirun command I am using to run mpi over IPoIB is,
mpirun --mca oob_tcp_if_include 192.168.1.0/24 --mca btl_tcp_include
10.1.0.0/24 --mca pml ob1 --mca btl tcp,sm,vader,self -hostfile nodes
-np 8 ./avg 8000
This program will appear to run on the nodes, but will sit at 100% CPU
and use no memory. On the host node an error will be printed,
[sm1][[58411,1],0][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
Using another program,
mpirun --mca oob_tcp_if_include 192.168.1.0/24 --mca btl_tcp_if_include
10.1.0.0/24 --mca pml ob1 --mca btl tcp,sm,vader,self -hostfile nodes
-np 8 ./congrad 800
Produces the following result. This program will also run on the nodes
sm1, sm2, sm3, and sm4 at 100% and use no memory.
[sm3][[61383,1],4][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.5 failed: No route to host (113)
[sm4][[61383,1],6][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.4 failed: No route to host (113)
[sm2][[61383,1],3][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.2 failed: No route to host (113)
[sm3][[61383,1],5][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.5 failed: No route to host (113)
[sm4][[61383,1],7][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.4 failed: No route to host (113)
[sm2][[61383,1],2][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.2 failed: No route to host (113)
[sm1][[61383,1],0][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
[sm1][[61383,1],1][btl_tcp_endpoint.c:803:mca_btl_tcp_endpoint_complete_connect]
connect() to 10.1.0.3 failed: No route to host (113)
*** openib ***
Running the congrad program over openib will produce the result,
mpirun --mca btl self,sm,openib --mca mtl ^psm --mca btl_tcp_if_include
10.1.0.0/24 -hostfile nodes -np 8 ./avg 800
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
--------------------------------------------------------------------------
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
Open MPI stopped checking at the first component that it did not find.
Host: sm2.overst.local
Framework: btl
Component: openib
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
mca_bml_base_open() failed
--> Returned "Not found" (-13) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[sm1.overst.local:32239] [[57506,0],1] usock_peer_send_blocking: send()
to socket 29 failed: Broken pipe (32)
[sm1.overst.local:32239] [[57506,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32239] [[57506,0],1]-[[57506,1],1] usock_peer_accept:
usock_peer_send_connect_ack failed
[sm1.overst.local:32239] [[57506,0],1] usock_peer_send_blocking: send()
to socket 27 failed: Broken pipe (32)
[sm1.overst.local:32239] [[57506,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32239] [[57506,0],1]-[[57506,1],0] usock_peer_accept:
usock_peer_send_connect_ack failed
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[smd:31760] 4 more processes have sent help message help-mca-base.txt /
find-available:not-valid
[smd:31760] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
[smd:31760] 4 more processes have sent help message help-mpi-runtime.txt
/ mpi_init:startup:internal-failure
=== Later errors printed out on the host node ===
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm3
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm1
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm2
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
------------------------------------------------------------
A process or daemon was unable to complete a TCP connection
to another process:
Local host: sm4
Remote host: 10.1.0.1
This is usually caused by a firewall on the remote host. Please
check that any firewall (e.g., iptables) has been disabled and
try again.
------------------------------------------------------------
The ./avg process was not created on any of the nodes.
Running the ./congrad program,
mpirun --mca btl self,sm,openib --mca mtl ^psm --mca btl_tcp_if_include
10.1.0.0/24 -hostfile nodes -np 8 ./congrad 800
Will results in the following errors,
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
--------------------------------------------------------------------------
A requested component was not found, or was unable to be opened. This
means that this component is either not installed or is unable to be
used on your system (e.g., sometimes this means that shared libraries
that the component requires are unable to be found/loaded). Note that
Open MPI stopped checking at the first component that it did not find.
Host: sm3.overst.local
Framework: btl
Component: openib
--------------------------------------------------------------------------
--------------------------------------------------------------------------
It looks like MPI_INIT failed for some reason; your parallel process is
likely to abort. There are many reasons that a parallel process can
fail during MPI_INIT; some of which are due to configuration or environment
problems. This failure appears to be an internal failure; here's some
additional information (which may only be relevant to an Open MPI
developer):
mca_bml_base_open() failed
--> Returned "Not found" (-13) instead of "Success" (0)
--------------------------------------------------------------------------
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[sm1.overst.local:32271] [[57834,0],1] usock_peer_send_blocking: send()
to socket 29 failed: Broken pipe (32)
[sm1.overst.local:32271] [[57834,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32271] [[57834,0],1]-[[57834,1],0] usock_peer_accept:
usock_peer_send_connect_ack failed
[sm1.overst.local:32271] [[57834,0],1] usock_peer_send_blocking: send()
to socket 27 failed: Broken pipe (32)
[sm1.overst.local:32271] [[57834,0],1] ORTE_ERROR_LOG: Unreachable in
file oob_usock_connection.c at line 316
[sm1.overst.local:32271] [[57834,0],1]-[[57834,1],1] usock_peer_accept:
usock_peer_send_connect_ack failed
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
*** An error occurred in MPI_Init
*** on a NULL communicator
*** MPI_ERRORS_ARE_FATAL (processes in this communicator will now abort,
*** and potentially your MPI job)
[smd:32088] 5 more processes have sent help message help-mca-base.txt /
find-available:not-valid
[smd:32088] Set MCA parameter "orte_base_help_aggregate" to 0 to see all
help / error messages
[smd:32088] 5 more processes have sent help message help-mpi-runtime.txt
/ mpi_init:startup:internal-failure
Using these mpirun commands will successfully run using a test program
that is only using point to point communication.
The nodes are interconnected in the following way.
__ _
[__]|=|
/::/|_|
HOST: smd
Dual 1Gb Ethernet Bonded
.-------------> Bond0 IP: 192.168.1.200
| Infiniband Card: MHQH29B-XTR <------------.
| Ib0 IP: 10.1.0.1 |
| OS: Ubuntu Mate |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: sm1 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.196 |
| Infiniband Card: QLOGIC QLE7340 <---------|
| Ib0 IP: 10.1.0.2 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
|---------. /::/|_| |
| | HOST: sm2 |
| | Dual 1Gb Ethernet Bonded |
| '---> Bond0 IP: 192.168.1.199 |
__________ Infiniband Card: QLOGIC QLE7340 __________
[_|||||||_°] Ib0 IP: 10.1.0.3 [_|||||||_°]
[_|||||||_°] OS: Centos 7 Minimal [_|||||||_°]
[_|||||||_°] __ _ [_|||||||_°]
Gb Ethernet Switch [__]|=| Voltaire 4036
QDR Switch
| /::/|_| |
| HOST: sm3 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.203 |
| Infiniband Card: QLOGIC QLE7340 <----------|
| Ib0 IP: 10.1.0.4 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: sm4 |
| Dual 1Gb Ethernet Bonded |
|-------------> Bond0 IP: 192.168.1.204 |
| Infiniband Card: QLOGIC QLE7340 <----------|
| Ib0 IP: 10.1.0.5 |
| OS: Centos 7 Minimal |
| __ _ |
| [__]|=| |
| /::/|_| |
| HOST: dl580 |
| Dual 1Gb Ethernet Bonded |
'-------------> Bond0 IP: 192.168.1.201 |
Infiniband Card: QLOGIC QLE7340 <----------'
Ib0 IP: 10.1.0.6
OS: Centos 7 Minimal
Thanks for the help again.
Sincerely,
Allan Overstreet