Discussion:
[OMPI users] communications groups
Marlborough, Rick
2016-10-14 21:33:28 UTC
Permalink
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an array of ints of size 1. The value it contains is 1. My comm world size is 5. The call to MPI_Barrier fails every time with error "invalid communicator". This code is pretty much copied out of a text book. I must be doing something wrong. I just don't see it. Can anyone else spot my error? I am using v2.01 on red hat 6.5.

Thanks
Rick


MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);

3.1.1001
George Bosilca
2016-10-14 21:43:43 UTC
Permalink
Rick,

Let's assume that you have started 2 processes, and that your sensorList is
{1}. The worldgroup will then be {P0, P1}, which trimmed via the sensorList
will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the sensorgroup {P1}
on P1. As a result on P0 you will create a MPI_COMM_NULL communicator,
while on P1 you will have a valid communicator sensorComm (which will only
contain P1). You cannot do a Barrier on an MPI_COMM_NULL communicator,
which might explain the "invalid communicator" error you are getting.

George.
Post by Marlborough, Rick
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an
array of ints of size 1. The value it contains is 1. My comm world size is
5. The call to MPI_Barrier fails every time with error “invalid
communicator”. This code is pretty much copied out of a text book. I must
be doing something wrong. I just don’t see it. Can anyone else spot my
error? I am using v2.01 on red hat 6.5.
Thanks
Rick
MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);
3.1.1001
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
Marlborough, Rick
2016-10-17 12:09:57 UTC
Permalink
Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little confusing. If my world group is P0,P1, visible on both processes, why wouldn't the sensorList contain P1 on both processes?

Rick


From: users [mailto:users-***@lists.open-mpi.org] On Behalf Of George Bosilca
Sent: Friday, October 14, 2016 5:44 PM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

Let's assume that you have started 2 processes, and that your sensorList is {1}. The worldgroup will then be {P0, P1}, which trimmed via the sensorList will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the sensorgroup {P1} on P1. As a result on P0 you will create a MPI_COMM_NULL communicator, while on P1 you will have a valid communicator sensorComm (which will only contain P1). You cannot do a Barrier on an MPI_COMM_NULL communicator, which might explain the "invalid communicator" error you are getting.

George.


On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick <***@aaccorp.com<mailto:***@aaccorp.com>> wrote:

Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an array of ints of size 1. The value it contains is 1. My comm world size is 5. The call to MPI_Barrier fails every time with error "invalid communicator". This code is pretty much copied out of a text book. I must be doing something wrong. I just don't see it. Can anyone else spot my error? I am using v2.01 on red hat 6.5.

Thanks
Rick


MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);

3.1.1001

_______________________________________________
users mailing list
***@lists.open-mpi.org<mailto:***@lists.open-mpi.org>
https://rfd.newmexicoconsortium.org/mailman/listinfo/users


3.1.1001
Gilles Gouaillardet
2016-10-17 13:30:25 UTC
Permalink
Rick,

I re-read the MPI standard and was unable to figure out if sensorgroup is
MPI_GROUP_EMPTY or a group with task 1 on tasks except task 1
(A group that does not contain the current task makes little sense to me,
but I do not see any reason why this group have to be MPI_GROUP_EMPTY)

Regardless, sensorComm will be MPI_COMM_NULL except on task 1, so
MPI_Barrier will fail.

Cheers,

Gilles
Post by Marlborough, Rick
Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little
confusing. If my world group is P0,P1, visible on both processes, why
wouldn’t the sensorList contain P1 on both processes?
Rick
Behalf Of *George Bosilca
*Sent:* Friday, October 14, 2016 5:44 PM
*To:* Open MPI Users
*Subject:* Re: [OMPI users] communications groups
Rick,
Let's assume that you have started 2 processes, and that your sensorList
is {1}. The worldgroup will then be {P0, P1}, which trimmed via the
sensorList will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the
sensorgroup {P1} on P1. As a result on P0 you will create a
MPI_COMM_NULL communicator, while on P1 you will have a valid communicator
sensorComm (which will only contain P1). You cannot do a Barrier on an
MPI_COMM_NULL communicator, which might explain the "invalid communicator"
error you are getting.
George.
On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick <
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an
array of ints of size 1. The value it contains is 1. My comm world size is
5. The call to MPI_Barrier fails every time with error “invalid
communicator”. This code is pretty much copied out of a text book. I must
be doing something wrong. I just don’t see it. Can anyone else spot my
error? I am using v2.01 on red hat 6.5.
Thanks
Rick
MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);
3.1.1001
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
3.1.1001
Marlborough, Rick
2016-10-17 14:10:01 UTC
Permalink
Designation: Non-Export Controlled Content
Gilles;
My scenario involves a Dispatcher of rank 0, and several sensors and proxy objects. The Dispatcher triggers activity and gathers results. The proxies get triggered first. They send data to the sensors, and the sensors indicate to the dispatcher that they are done. I am trying to create 2 comm groups. One for the sensors and one for the proxies. The dispatcher will use the 2 comm groups to coordinate activity. I tried adding the dispatcher to the sensorList comm group, but I get an error saying "invalid task".

Rick

From: users [mailto:users-***@lists.open-mpi.org] On Behalf Of Gilles Gouaillardet
Sent: Monday, October 17, 2016 9:30 AM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

I re-read the MPI standard and was unable to figure out if sensorgroup is MPI_GROUP_EMPTY or a group with task 1 on tasks except task 1
(A group that does not contain the current task makes little sense to me, but I do not see any reason why this group have to be MPI_GROUP_EMPTY)

Regardless, sensorComm will be MPI_COMM_NULL except on task 1, so MPI_Barrier will fail.

Cheers,

Gilles

On Monday, October 17, 2016, Marlborough, Rick <***@aaccorp.com<mailto:***@aaccorp.com>> wrote:

Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little confusing. If my world group is P0,P1, visible on both processes, why wouldn't the sensorList contain P1 on both processes?

Rick


From: users [mailto:users-***@lists.open-mpi.org<javascript:_e(%7B%7D,'cvml','users-***@lists.open-mpi.org');>] On Behalf Of George Bosilca
Sent: Friday, October 14, 2016 5:44 PM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

Let's assume that you have started 2 processes, and that your sensorList is {1}. The worldgroup will then be {P0, P1}, which trimmed via the sensorList will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the sensorgroup {P1} on P1. As a result on P0 you will create a MPI_COMM_NULL communicator, while on P1 you will have a valid communicator sensorComm (which will only contain P1). You cannot do a Barrier on an MPI_COMM_NULL communicator, which might explain the "invalid communicator" error you are getting.

George.


On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick <***@aaccorp.com<javascript:_e(%7B%7D,'cvml','***@aaccorp.com');>> wrote:

Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an array of ints of size 1. The value it contains is 1. My comm world size is 5. The call to MPI_Barrier fails every time with error "invalid communicator". This code is pretty much copied out of a text book. I must be doing something wrong. I just don't see it. Can anyone else spot my error? I am using v2.01 on red hat 6.5.

Thanks
Rick


MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);

3.1.1001

_______________________________________________
users mailing list
***@lists.open-mpi.org<javascript:_e(%7B%7D,'cvml','***@lists.open-mpi.org');>
https://rfd.newmexicoconsortium.org/mailman/listinfo/users


3.1.1001

3.1.1001
Gilles Gouaillardet
2016-10-17 14:37:42 UTC
Permalink
Rick,

So you have three types of tasks
- 1 dispatcher
- several sensors
- several proxies

If proxies do not communicate with each other, and if sensors do not
communicate with each other, then you could end up with 3 inter
communicators
sensorComm: dispatcher in the left group and sensors in the right group
proxyComm: dispatcher in the left group and proxies in the right group
controlComm: sensors in the left group and proxies in the right group

Does that fit your needs ?
If yes, then keep in mind sensorComm is MPI_COMM_NULL on the proxy tasks,
proxyComm is MPI_COMM_NULL on the sensor tasks, and controlComm is
MPI_COMM_NULL on the dispatcher.

Cheers,

Gilles
Post by Marlborough, Rick
Designation: Non-Export Controlled Content
Gilles;
My scenario involves a Dispatcher of rank 0, and several
sensors and proxy objects. The Dispatcher triggers activity and gathers
results. The proxies get triggered first. They send data to the sensors,
and the sensors indicate to the dispatcher that they are done. I am trying
to create 2 comm groups. One for the sensors and one for the proxies. The
dispatcher will use the 2 comm groups to coordinate activity. I tried
adding the dispatcher to the sensorList comm group, but I get an error
saying “invalid task”.
Rick
Behalf Of *Gilles Gouaillardet
*Sent:* Monday, October 17, 2016 9:30 AM
*To:* Open MPI Users
*Subject:* Re: [OMPI users] communications groups
Rick,
I re-read the MPI standard and was unable to figure out if sensorgroup is
MPI_GROUP_EMPTY or a group with task 1 on tasks except task 1
(A group that does not contain the current task makes little sense to me,
but I do not see any reason why this group have to be MPI_GROUP_EMPTY)
Regardless, sensorComm will be MPI_COMM_NULL except on task 1, so MPI_Barrier will fail.
Cheers,
Gilles
Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little
confusing. If my world group is P0,P1, visible on both processes, why
wouldn’t the sensorList contain P1 on both processes?
Rick
Bosilca
*Sent:* Friday, October 14, 2016 5:44 PM
*To:* Open MPI Users
*Subject:* Re: [OMPI users] communications groups
Rick,
Let's assume that you have started 2 processes, and that your sensorList
is {1}. The worldgroup will then be {P0, P1}, which trimmed via the
sensorList will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the
sensorgroup {P1} on P1. As a result on P0 you will create a
MPI_COMM_NULL communicator, while on P1 you will have a valid communicator
sensorComm (which will only contain P1). You cannot do a Barrier on an
MPI_COMM_NULL communicator, which might explain the "invalid communicator"
error you are getting.
George.
On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick <
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an
array of ints of size 1. The value it contains is 1. My comm world size is
5. The call to MPI_Barrier fails every time with error “invalid
communicator”. This code is pretty much copied out of a text book. I must
be doing something wrong. I just don’t see it. Can anyone else spot my
error? I am using v2.01 on red hat 6.5.
Thanks
Rick
MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);
3.1.1001
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
3.1.1001
3.1.1001
Marlborough, Rick
2016-10-17 15:43:04 UTC
Permalink
Designation: Non-Export Controlled Content
Gilles;
Yes, your assumption is correct. No communication between proxies and no communications between sensors. I am using rank to determine role. Dispatcher being 0. Sensors start at 1. So I should have 3 groups? I am new to MPI and my knowledge of it is not the best. My understanding is that when I utilize an MPI_Barrier, all participants of a specified group must call MPI_Barrier in order to advance. This tells me Dispatch and sensor must be in the same group. Is my understanding incorrect?

Thanks
Rick

From: users [mailto:users-***@lists.open-mpi.org] On Behalf Of Gilles Gouaillardet
Sent: Monday, October 17, 2016 10:38 AM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

So you have three types of tasks
- 1 dispatcher
- several sensors
- several proxies

If proxies do not communicate with each other, and if sensors do not communicate with each other, then you could end up with 3 inter communicators
sensorComm: dispatcher in the left group and sensors in the right group
proxyComm: dispatcher in the left group and proxies in the right group
controlComm: sensors in the left group and proxies in the right group

Does that fit your needs ?
If yes, then keep in mind sensorComm is MPI_COMM_NULL on the proxy tasks, proxyComm is MPI_COMM_NULL on the sensor tasks, and controlComm is MPI_COMM_NULL on the dispatcher.

Cheers,

Gilles

On Monday, October 17, 2016, Marlborough, Rick <***@aaccorp.com<mailto:***@aaccorp.com>> wrote:

Designation: Non-Export Controlled Content
Gilles;
My scenario involves a Dispatcher of rank 0, and several sensors and proxy objects. The Dispatcher triggers activity and gathers results. The proxies get triggered first. They send data to the sensors, and the sensors indicate to the dispatcher that they are done. I am trying to create 2 comm groups. One for the sensors and one for the proxies. The dispatcher will use the 2 comm groups to coordinate activity. I tried adding the dispatcher to the sensorList comm group, but I get an error saying "invalid task".

Rick

From: users [mailto:users-***@lists.open-mpi.org<javascript:_e(%7B%7D,'cvml','users-***@lists.open-mpi.org');>] On Behalf Of Gilles Gouaillardet
Sent: Monday, October 17, 2016 9:30 AM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

I re-read the MPI standard and was unable to figure out if sensorgroup is MPI_GROUP_EMPTY or a group with task 1 on tasks except task 1
(A group that does not contain the current task makes little sense to me, but I do not see any reason why this group have to be MPI_GROUP_EMPTY)

Regardless, sensorComm will be MPI_COMM_NULL except on task 1, so MPI_Barrier will fail.

Cheers,

Gilles

On Monday, October 17, 2016, Marlborough, Rick <***@aaccorp.com<javascript:_e(%7B%7D,'cvml','***@aaccorp.com');>> wrote:

Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little confusing. If my world group is P0,P1, visible on both processes, why wouldn't the sensorList contain P1 on both processes?

Rick


From: users [mailto:users-***@lists.open-mpi.org] On Behalf Of George Bosilca
Sent: Friday, October 14, 2016 5:44 PM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups

Rick,

Let's assume that you have started 2 processes, and that your sensorList is {1}. The worldgroup will then be {P0, P1}, which trimmed via the sensorList will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the sensorgroup {P1} on P1. As a result on P0 you will create a MPI_COMM_NULL communicator, while on P1 you will have a valid communicator sensorComm (which will only contain P1). You cannot do a Barrier on an MPI_COMM_NULL communicator, which might explain the "invalid communicator" error you are getting.

George.


On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick <***@aaccorp.com<mailto:***@aaccorp.com>> wrote:

Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an array of ints of size 1. The value it contains is 1. My comm world size is 5. The call to MPI_Barrier fails every time with error "invalid communicator". This code is pretty much copied out of a text book. I must be doing something wrong. I just don't see it. Can anyone else spot my error? I am using v2.01 on red hat 6.5.

Thanks
Rick


MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);

3.1.1001

_______________________________________________
users mailing list
***@lists.open-mpi.org<mailto:***@lists.open-mpi.org>
https://rfd.newmexicoconsortium.org/mailman/listinfo/users


3.1.1001

3.1.1001

3.1.1001
Gilles Gouaillardet
2016-10-18 11:38:54 UTC
Permalink
Rick,

if you look at the big picture, i think 3 communicators make more sense.
now keep in mind that a given task is only part of two valid communicators.
depending on how you implement communicator creation, there could be 2
communicators per task,
or 3 communicators, and one is MPI_COMM_NULL.

MPI_Barrier is invoked on communicators, not group.
all members of a communicator must call MPI_Barrier().
now keep in mind that when you create a communicator, it might return
MPI_COMM_NULL.
this is a special case, and you are not allowed to MPI_Barrier(MPI_COMM_NULL)


Cheers,

Gilles

On Tue, Oct 18, 2016 at 12:43 AM, Marlborough, Rick
Post by Marlborough, Rick
Designation: Non-Export Controlled Content
Gilles;
Yes, your assumption is correct. No communication between
proxies and no communications between sensors. I am using rank to determine
role. Dispatcher being 0. Sensors start at 1. So I should have 3 groups? I
am new to MPI and my knowledge of it is not the best. My understanding is
that when I utilize an MPI_Barrier, all participants of a specified group
must call MPI_Barrier in order to advance. This tells me Dispatch and sensor
must be in the same group. Is my understanding incorrect?
Thanks
Rick
Sent: Monday, October 17, 2016 10:38 AM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups
Rick,
So you have three types of tasks
- 1 dispatcher
- several sensors
- several proxies
If proxies do not communicate with each other, and if sensors do not
communicate with each other, then you could end up with 3 inter
communicators
sensorComm: dispatcher in the left group and sensors in the right group
proxyComm: dispatcher in the left group and proxies in the right group
controlComm: sensors in the left group and proxies in the right group
Does that fit your needs ?
If yes, then keep in mind sensorComm is MPI_COMM_NULL on the proxy tasks,
proxyComm is MPI_COMM_NULL on the sensor tasks, and controlComm is
MPI_COMM_NULL on the dispatcher.
Cheers,
Gilles
Designation: Non-Export Controlled Content
Gilles;
My scenario involves a Dispatcher of rank 0, and several
sensors and proxy objects. The Dispatcher triggers activity and gathers
results. The proxies get triggered first. They send data to the sensors, and
the sensors indicate to the dispatcher that they are done. I am trying to
create 2 comm groups. One for the sensors and one for the proxies. The
dispatcher will use the 2 comm groups to coordinate activity. I tried adding
the dispatcher to the sensorList comm group, but I get an error saying
“invalid task”.
Rick
Sent: Monday, October 17, 2016 9:30 AM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups
Rick,
I re-read the MPI standard and was unable to figure out if sensorgroup is
MPI_GROUP_EMPTY or a group with task 1 on tasks except task 1
(A group that does not contain the current task makes little sense to me,
but I do not see any reason why this group have to be MPI_GROUP_EMPTY)
Regardless, sensorComm will be MPI_COMM_NULL except on task 1, so MPI_Barrier will fail.
Cheers,
Gilles
Designation: Non-Export Controlled Content
George;
Thanks for your response. Your second sentence is a little
confusing. If my world group is P0,P1, visible on both processes, why
wouldn’t the sensorList contain P1 on both processes?
Rick
Sent: Friday, October 14, 2016 5:44 PM
To: Open MPI Users
Subject: Re: [OMPI users] communications groups
Rick,
Let's assume that you have started 2 processes, and that your sensorList is
{1}. The worldgroup will then be {P0, P1}, which trimmed via the sensorList
will give the sensorgroup {MPI_GROUP_EMPTY} on P0 and the sensorgroup {P1}
on P1. As a result on P0 you will create a MPI_COMM_NULL communicator, while
on P1 you will have a valid communicator sensorComm (which will only contain
P1). You cannot do a Barrier on an MPI_COMM_NULL communicator, which might
explain the "invalid communicator" error you are getting.
George.
On Fri, Oct 14, 2016 at 5:33 PM, Marlborough, Rick
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an array
of ints of size 1. The value it contains is 1. My comm world size is 5. The
call to MPI_Barrier fails every time with error “invalid communicator”. This
code is pretty much copied out of a text book. I must be doing something
wrong. I just don’t see it. Can anyone else spot my error? I am using v2.01
on red hat 6.5.
Thanks
Rick
MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);
3.1.1001
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
3.1.1001
3.1.1001
3.1.1001
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
Gilles Gouaillardet
2016-10-17 13:24:02 UTC
Permalink
Rick,

In my understanding, sensorgroup is a group with only task 1
Consequently, sensorComm is
- similar to MPI_COMM_SELF on task 1
- MPI_COMM_NULL on other tasks, and hence the barrier fails

I suggest you double check sensorgroup is never MPI_GROUP_EMPTY
and add a test not to call MPI_Barrier on MPI_COMM_NULL

Makes sense ?

Cheers,

Gilles
Post by Marlborough, Rick
Designation: Non-Export Controlled Content
Folks;
I have the following code setup. The sensorList is an
array of ints of size 1. The value it contains is 1. My comm world size is
5. The call to MPI_Barrier fails every time with error “invalid
communicator”. This code is pretty much copied out of a text book. I must
be doing something wrong. I just don’t see it. Can anyone else spot my
error? I am using v2.01 on red hat 6.5.
Thanks
Rick
MPI_Comm_group(MPI_COMM_WORLD, &worldgroup);
MPI_Group_incl(worldgroup, 1, sensorList, &sensorgroup);
MPI_Comm_create(MPI_COMM_WORLD, sensorgroup, &sensorComm);
MPI_Barrier(sensorComm);
3.1.1001
Loading...