[OMPI users] hwloc, OpenMPI and unsupported OSes and toolchains

[Tim Kelly]

Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain. I have a
project as part of my PhD work, and it's not a simple, one-part
question. For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported. If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization. That's part
of what I am trying to determine with my inquiry, and not a reflection
on OpenMPI.

To summarize some of the question areas:

1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable

This is a fairly dedicated system to be used for analyzing ODEs (disease
models). The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).

If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.

Thank you,
tim

[Brice Goglin]

Hello

I am available for off-line discussion for the hwloc side of things. But
things look complicated here from your summary below. I guess there's no
need for binding on such a system. And topology is quite simple, so it
might be easier to hardwire everything.

Brice

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain.  I have a
project as part of my PhD work, and it's not a simple, one-part
question.  For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported.  If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization.  That's part
of what I am trying to determine with my inquiry, and not a reflection
on OpenMPI.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease
models).  The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.
Thank you,
tim

[Tim Kelly]

Hi Brice,
Thank you for your response. Yes, possibly hardwiring everything would
be easier. I was thinking I could use OpenMPI for the signaling between
the cores on an OS that doesn't support multi-processing, using the
shmem approach. The executable is the same for each CPU image, but the
initial conditions will be different and have to be communicated from
the master CPU, which will also be responsible for aggregating the
results into a data and image file.

Here's the hardware:

https://shop.udoo.org/other/x86/udoo-x86-advanced-plus.html

The OS running on CPU0 would have control of I/O and partition the
memory. In testing to this point 1GB is sufficient to run the OS and
executable. In fact, currently I do not free any memory so likely the
footprint is much, much smaller. Not freeing the memory until execution
is completed has avoided some malloc complaints. The OS does not have
paging or segmentation, so fragmentation can be an issue. Performance
is very good.

Is is possible to use a shared memory approach and run an AMP set up,
with hwloc? Would there be any benefit to doing so instead of the
hardwiring approach you mention?

tim

Post by Brice Goglin
Hello
I am available for off-line discussion for the hwloc side of things. But
things look complicated here from your summary below. I guess there's no
need for binding on such a system. And topology is quite simple, so it
might be easier to hardwire everything.
Brice

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain. I have a
project as part of my PhD work, and it's not a simple, one-part
question. For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported. If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization. That's part
of what I am trying to determine with my inquiry, and not a reflection
on OpenMPI.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease
models). The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.
Thank you,
tim

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Gregory (tim) Kelly]

Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain. I have a
project as part of my PhD work, and it's not a simple, one-part
question. For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported. If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization. That's part
of what I am trying to determine with my inquiry.

To summarize some of the question areas:

1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable

This is a fairly dedicated system to be used for analyzing ODEs (disease
models). The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).

If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.

Thank you,
tim

[Ralph H Castain]

I don’t see how Open MPI can operate without pthreads

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions about hwloc, OpenMPI and an alternative OS and toolchain. I have a project as part of my PhD work, and it's not a simple, one-part question. For brevity, I am omitting details about the OS and toolchain, other than that neither are supported. If forced to choose between OpenMPI and the OS/toolchain, I am likely to choose the OS/toolchain and pursue other avenues for parallelization. That's part of what I am trying to determine with my inquiry.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an asymmetric multi-processing system where only one core has I/O but the other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease models). The hardware is cheap ($200) and uses very little power (can run off a 12v battery), and the toolchain and OS are all BSD-licensed (and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary traffic to the list), I'd be more than willing to summarize the conversation and contribute it to the online documentation.
Thank you,
tim
--
"Nuclear power is a hell of a way to boil water." -- Albert Einstein
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Jeff Squyres (jsquyres)]

To follow up for the web thread: I talked with Tim about this off-list.

Not only will Open MPI likely not work in Tim's environment, MPI itself is probably too much for what he's trying to do.

Post by Ralph H Castain
I don’t see how Open MPI can operate without pthreads

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions about hwloc, OpenMPI and an alternative OS and toolchain. I have a project as part of my PhD work, and it's not a simple, one-part question. For brevity, I am omitting details about the OS and toolchain, other than that neither are supported. If forced to choose between OpenMPI and the OS/toolchain, I am likely to choose the OS/toolchain and pursue other avenues for parallelization. That's part of what I am trying to determine with my inquiry.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an asymmetric multi-processing system where only one core has I/O but the other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease models). The hardware is cheap ($200) and uses very little power (can run off a 12v battery), and the toolchain and OS are all BSD-licensed (and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary traffic to the list), I'd be more than willing to summarize the conversation and contribute it to the online documentation.
Thank you,
tim
--
"Nuclear power is a hell of a way to boil water." -- Albert Einstein
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

--
Jeff Squyres
***@cisco.com

[Tim Kelly]

Yes, I was going to aggregate some of the conversations that I have had
(four of them, thank you much for the responses). The consensus was
that neither hwloc nor OpenMPI were appropriate for my project.

The use of shared memory was proposed by Jeff Squyres and one other
person. From there is it a matter of deciding between interrupt driven
messaging or semaphores. This (apparently unpublished) paper discusses
the pros and cons of each:

http://cs.brown.edu/people/irina/papers/2013-opodis.pdf

One of the first approaches they discuss is Barrelfish OS, which looks
to be comparable to an exokernel running an image of the OS on each CPU
node:

http://www.barrelfish.org/

This appears to have a robust shared memory approach. I'm still
digesting the details, but it looks to solve many of the problems I am
looking at.

Thanks again for the discussions!

tim

Post by Jeff Squyres (jsquyres)
To follow up for the web thread: I talked with Tim about this off-list.
Not only will Open MPI likely not work in Tim's environment, MPI itself is probably too much for what he's trying to do.

Post by Ralph H Castain
I don’t see how Open MPI can operate without pthreads

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions about hwloc, OpenMPI and an alternative OS and toolchain. I have a project as part of my PhD work, and it's not a simple, one-part question. For brevity, I am omitting details about the OS and toolchain, other than that neither are supported. If forced to choose between OpenMPI and the OS/toolchain, I am likely to choose the OS/toolchain and pursue other avenues for parallelization. That's part of what I am trying to determine with my inquiry.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an asymmetric multi-processing system where only one core has I/O but the other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease models). The hardware is cheap ($200) and uses very little power (can run off a 12v battery), and the toolchain and OS are all BSD-licensed (and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary traffic to the list), I'd be more than willing to summarize the conversation and contribute it to the online documentation.
Thank you,
tim
--
"Nuclear power is a hell of a way to boil water." -- Albert Einstein
_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

_______________________________________________
users mailing list
https://lists.open-mpi.org/mailman/listinfo/users

[Gregory (tim) Kelly]

Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain. I have a
project as part of my PhD work, and it's not a simple, one-part
question. For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported. If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization. That's part
of what I am trying to determine with my inquiry, and not a reflection
on OpenMPI.

To summarize some of the question areas:

1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable

This is a fairly dedicated system to be used for analyzing ODEs (disease
models). The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).

If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.

Thank you,
tim

[Tim Kelly]

My apologies for the duplicate posts. This was originally sent from my
university email account. However, there was a configuration error at
the mailing list host site that prevented email exchange between the
host and my university, including confirmation that I had subscribed.
Jeff Squyres manually subscribed me, and when a second send of this post
did not go through, the mailing list host and my university's IT
department figured it out. Once that was fixed these emails from my
university account came through, after being held in queue. In the
meantime I had switched to a different email account and sent the post.

Post by Tim Kelly
Hello Everyone,
I'm inquiring to find someone that can answer some multi-part questions
about hwloc, OpenMPI and an alternative OS and toolchain. I have a
project as part of my PhD work, and it's not a simple, one-part
question. For brevity, I am omitting details about the OS and
toolchain, other than that neither are supported. If forced to choose
between OpenMPI and the OS/toolchain, I am likely to choose the
OS/toolchain and pursue other avenues for parallelization. That's part
of what I am trying to determine with my inquiry, and not a reflection
on OpenMPI.
1) The OS I am working with does not support MP
2) nor does it support pthreads
3) the hardware is quad-core x86 SoC with an integrated memory controller
4) I'd like to see if it possible to utilize hwloc and shmem to build an
asymmetric multi-processing system where only one core has I/O but the
other three can run the executable
This is a fairly dedicated system to be used for analyzing ODEs (disease
models). The hardware is cheap ($200) and uses very little power (can
run off a 12v battery), and the toolchain and OS are all BSD-licensed
(and everything will be published under that license).
If someone is available for off-line discussion (to minimize unnecessary
traffic to the list), I'd be more than willing to summarize the
conversation and contribute it to the online documentation.
Thank you,
tim