[OMPI users] How to use MPI_Win_attach() (or how to specify the 'displ' on a remote process)

[Clune, Thomas L. (GSFC-6101)]

I have encountered a problem that seems well suited to dynamic windows with one-sided comunication. To verify my understanding, I put together a simple demo code (attached). My initial attempt consistently crashed until I stumbled upon passing the base address of the attached memory on the _target_ process to the sending process in advance of the MPI_Put() call. Previously, I’d been specifying a displacement of 0 in the put.

The part I find confusing is that essentially the same example using MPI_Win_allocate() did _not_ require passing the displacement. I.e., MPI_Put() used a displacement of zero and my data ended up where I expected it on the target process. This would suggest that MPI_Put() must be used differently depending on whether or not a window is dynamic. I did not find this mentioned anywhere in the documentation, nor did I find any example code that demonstrated this extra communication for the displacement.

OTOH, this does help to explain another question I had about dynamic windows. The documentation states that there can me multiple (nonoverlapping) memory regions attached to a window, but does not mention how to target specific regions.

If I’m right about all of this, then I guess this is just a request for the relevant parties to improve the documentation and the various examples found on the web. If I’m wrong, then I’d appreciate enlightenment.

I apologize in advance that the attached code is in Fortran (my native tongue). The code runs fine as-is. But if the commented out MPI_Put() call is substituted for the uncommented one, it will crash with a segmentation fault.

[Nathan Hjelm]

This behavior is clearly specified in the standard. From MPI 3.1 § 11.2.4:

In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the target_disp for all RMA functions is the address at the target; i.e., the effective window_base is MPI_BOTTOM and the disp_unit is one. For dynamic windows, the target_disp argument to RMA communication operations is not restricted to non-negative values. Users should use MPI_GET_ADDRESS at the target process to determine the address of a target memory location and communicate this address to the origin process.


So by using a target address of 0 you are effectively trying to write to NULL. osc/pt2pt's bounds checking isn't perfect so it is giving a SEGV.

-Nathan

On May 04, 2017, at 03:18 PM, "Clune, Thomas L. (GSFC-6101)" <***@nasa.gov> wrote:

I have encountered a problem that seems well suited to dynamic windows with one-sided comunication. To verify my understanding, I put together a simple demo code (attached). My initial attempt consistently crashed until I stumbled upon passing the base address of the attached memory on the _target_ process to the sending process in advance of the MPI_Put() call. Previously, I’d been specifying a displacement of 0 in the put.

The part I find confusing is that essentially the same example using MPI_Win_allocate() did _not_ require passing the displacement. I.e., MPI_Put() used a displacement of zero and my data ended up where I expected it on the target process. This would suggest that MPI_Put() must be used differently depending on whether or not a window is dynamic. I did not find this mentioned anywhere in the documentation, nor did I find any example code that demonstrated this extra communication for the displacement.

OTOH, this does help to explain another question I had about dynamic windows. The documentation states that there can me multiple (nonoverlapping) memory regions attached to a window, but does not mention how to target specific regions.

If I’m right about all of this, then I guess this is just a request for the relevant parties to improve the documentation and the various examples found on the web. If I’m wrong, then I’d appreciate enlightenment.

I apologize in advance that the attached code is in Fortran (my native tongue). The code runs fine as-is. But if the commented out MPI_Put() call is substituted for the uncommented one, it will crash with a segmentation fault.

[Clune, Thomas L. (GSFC-6101)]

On May 4, 2017, at 5:36 PM, Nathan Hjelm <***@me.com<mailto:***@me.com>> wrote:

This behavior is clearly specified in the standard. From MPI 3.1 § 11.2.4:

Thanks - I see it now. Some of the text is so similar to the online man pages that I must have glossed over that critical phrase. It remains unfortunate that every online example I have found for using dynamic windows is fundamentally flawed in this manner.

Am I correct then that a safe approach to using MPI_Put() would require a check on the MPI_WIN_CREATE_FLAVOR attribute and ensure the exchange of target addresses in the case of a dynamic window? I suppose most libraries have the put() in the same layer that creates the window, which sidesteps this concern.

Cheers,

- Tom






In the case of a window created with MPI_WIN_CREATE_DYNAMIC, the target_disp for all RMA functions is the address at the target; i.e., the effective window_base is MPI_BOTTOM and the disp_unit is one. For dynamic windows, the target_disp argument to RMA communication operations is not restricted to non-negative values. Users should use MPI_GET_ADDRESS at the target process to determine the address of a target memory location and communicate this address to the origin process.


So by using a target address of 0 you are effectively trying to write to NULL. osc/pt2pt's bounds checking isn't perfect so it is giving a SEGV.

-Nathan

On May 04, 2017, at 03:18 PM, "Clune, Thomas L. (GSFC-6101)" <***@nasa.gov<mailto:***@nasa.gov>> wrote:

I have encountered a problem that seems well suited to dynamic windows with one-sided comunication. To verify my understanding, I put together a simple demo code (attached). My initial attempt consistently crashed until I stumbled upon passing the base address of the attached memory on the _target_ process to the sending process in advance of the MPI_Put() call. Previously, I’d been specifying a displacement of 0 in the put.

The part I find confusing is that essentially the same example using MPI_Win_allocate() did _not_ require passing the displacement. I.e., MPI_Put() used a displacement of zero and my data ended up where I expected it on the target process. This would suggest that MPI_Put() must be used differently depending on whether or not a window is dynamic. I did not find this mentioned anywhere in the documentation, nor did I find any example code that demonstrated this extra communication for the displacement.

OTOH, this does help to explain another question I had about dynamic windows. The documentation states that there can me multiple (nonoverlapping) memory regions attached to a window, but does not mention how to target specific regions.

If I’m right about all of this, then I guess this is just a request for the relevant parties to improve the documentation and the various examples found on the web. If I’m wrong, then I’d appreciate enlightenment.

I apologize in advance that the attached code is in Fortran (my native tongue). The code runs fine as-is. But if the commented out MPI_Put() call is substituted for the uncommented one, it will crash with a segmentation fault.



_______________________________________________
users mailing list
***@lists.open-mpi.org<mailto:***@lists.open-mpi.org>
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
<mpi_win_dynamic.F90>_______________________________________________
users mailing list
***@lists.open-mpi.org<mailto:***@lists.open-mpi.org>
https://rfd.newmexicoconsortium.org/mailman/listinfo/users