[OMPI users] How to configure the message size in openMPI (over RDMA)?

Gilles Gouaillardet

2017-07-19 00:13:18 UTC

Hi,

i cannot comment for the openib specific part.

the coll/tuned collective module is very likely to split messages in
order to use a more efficient

algorithm. an other way to put it is you probably do not want to use
large messages.

but if this is really what you want, then one option is to disable
coll/tuned,

mpirun --mca coll ^tuned ...

an other option is to force the the algorithm used by coll/tuned

mpirun --mca coll_tuned_use_dynamic_rules 1 --mca
coll_tuned_allreduce_algorithm <algo> ...

where <algo> is a number between 1 and 6 (see ompi_info --all for the
definition)

Cheers,

Gilles

Post by Juncheng Gu
Hi,
I am trying to setup openMPI over RDMA cross machines.
I call MPI_AllReduce() with a 240MB data buffer.
But, it seems openMPI chunks data into small fragments (1MB ~ 15MB),
and then sends them out through RDMA.
Which mca parameters can affect the message size in openMPI?
How to configure "mca" to let openMPI use large message size
(fragment) in data transmission?
For example, is there any way we can set the mini message size of openMPI?
I have followed the instructions from
https://www.open-mpi.org/faq/?category=openfabrics#ofed-and-ompi. But
there is no improvement in message size.
I am using " --mca btl self,sm,openib ".
Best,
Juncheng
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users