Gilles Gouaillardet
2016-10-27 09:19:02 UTC
Jeff,
Out of curiosity, did you compile the Fortran test program with -O0 ?
Cheers,
Gilles
Out of curiosity, did you compile the Fortran test program with -O0 ?
Cheers,
Gilles
Jeff,
Thanks for looking at this. I know it isn't specific to Open-MPI, but it is a frustrating issue vis-a-vis MPI and Fortran. There are many very large MPI applications around the world written in Fortran that could benefit greatly from this MPI-3 capability. My own background is in numerical weather prediction, and I know it would be welcome in that community. Someone knowledgeable in both C and Fortran should be able to get to bottom of it.
T. Rosmond
I can reproduce this with Intel 17 and MPICH on Mac OSX so it's not an Open-MPI issue. I added VOLATILE to the shared memory objects to prevent Fortran compiler optimizations as well as a bunch of MPI_Win_sync calls (after replacing fence with lock_all/unlock_all), but neither changed the outcome.
While I claim a modest understanding of MPI-3 RMA and Fortran 2008, unfortunately, I have never figured out how to use MPI-3 shared memory from Fortran, which is especially unfortunate since it seems to be a fantastic source of frustration to both real users such as yourself and MPI+Fortran standard experts (Rolf).
Sorry for the unsatisfying response, but my suspicion is that this is a program correctness issue. I can't point to any error, but I've ruled out the obvious alternatives.
Jeff
I am trying to understand the use of the shared memory features of MPI-3 that allow direct sharing of the memory space of on-node processes. Attached are 2 small test programs, one written in C (testmpi3.c), the other F95 (testmpi3.f90) . They are solving the identical 'halo' exchange problem. 'testmpi3.c' is a simplified version of an example program from a presentation by Mark Lubin of Intel. I wrote 'testmpi3.f90' to mimic the C version.
CC_testmpi3.txt
F95_testmpi3.txt
Note: All 4 files are contained in the attached 'testmpi3.tar.gz'.
Comparing the outputs of each version, it is clear that the shared memory copies in 'testmpi3.c' are working correctly, but not in 'testmpi3.f90'. As far as I can tell, the 2 programs are equivalent up to line 134 of 'testmpi3.c' and lines 97-101 of 'testmpi3.f90'. I thought the calls to 'c_f_pointer' would produce Fortran pointers that would access the correct shared memory addresses as the C-pointers do in 'testmpi3.c', but clearly that isn't happening. Can anyone explain why not, and what is needed to make this happen. Any suggestions are welcome.
 Scientific Linux 6.8
 INTEL FORTRAN and ICC version 15.0.2.164
 OPEN-MPI 2.0.1
T. Rosmond
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
--
Jeff Hammond
http://jeffhammond.github.io/
Thanks for looking at this. I know it isn't specific to Open-MPI, but it is a frustrating issue vis-a-vis MPI and Fortran. There are many very large MPI applications around the world written in Fortran that could benefit greatly from this MPI-3 capability. My own background is in numerical weather prediction, and I know it would be welcome in that community. Someone knowledgeable in both C and Fortran should be able to get to bottom of it.
T. Rosmond
I can reproduce this with Intel 17 and MPICH on Mac OSX so it's not an Open-MPI issue. I added VOLATILE to the shared memory objects to prevent Fortran compiler optimizations as well as a bunch of MPI_Win_sync calls (after replacing fence with lock_all/unlock_all), but neither changed the outcome.
While I claim a modest understanding of MPI-3 RMA and Fortran 2008, unfortunately, I have never figured out how to use MPI-3 shared memory from Fortran, which is especially unfortunate since it seems to be a fantastic source of frustration to both real users such as yourself and MPI+Fortran standard experts (Rolf).
Sorry for the unsatisfying response, but my suspicion is that this is a program correctness issue. I can't point to any error, but I've ruled out the obvious alternatives.
Jeff
I am trying to understand the use of the shared memory features of MPI-3 that allow direct sharing of the memory space of on-node processes. Attached are 2 small test programs, one written in C (testmpi3.c), the other F95 (testmpi3.f90) . They are solving the identical 'halo' exchange problem. 'testmpi3.c' is a simplified version of an example program from a presentation by Mark Lubin of Intel. I wrote 'testmpi3.f90' to mimic the C version.
CC_testmpi3.txt
F95_testmpi3.txt
Note: All 4 files are contained in the attached 'testmpi3.tar.gz'.
Comparing the outputs of each version, it is clear that the shared memory copies in 'testmpi3.c' are working correctly, but not in 'testmpi3.f90'. As far as I can tell, the 2 programs are equivalent up to line 134 of 'testmpi3.c' and lines 97-101 of 'testmpi3.f90'. I thought the calls to 'c_f_pointer' would produce Fortran pointers that would access the correct shared memory addresses as the C-pointers do in 'testmpi3.c', but clearly that isn't happening. Can anyone explain why not, and what is needed to make this happen. Any suggestions are welcome.
 Scientific Linux 6.8
 INTEL FORTRAN and ICC version 15.0.2.164
 OPEN-MPI 2.0.1
T. Rosmond
_______________________________________________
users mailing list
https://rfd.newmexicoconsortium.org/mailman/listinfo/users
--
Jeff Hammond
http://jeffhammond.github.io/