Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2153 Discussions

Can shared memory work between processes running w/wo mpiexec?

Lei_T_
Beginner
467 Views

I have a GUI.exe (with GUI) and engine.exe (without GUI). I am using shared memory for inter-process communication between these two exe. Everything was working fine before I use mpiexec. After I added mpiexec for engine.exe, these two processes cannot talk to each other through shared memory. It seems that the shared memory is "shield" by mpiexe and cannot be shared by outside world, since mpiexec itself is also using shared memory for communication for parallel computations.

By the way, these two exe run on the same PC.

Is there any way to overcome this problem?

0 Kudos
5 Replies
TimP
Honored Contributor III
467 Views

In MPI-3 there's a new shared memory feature which ought to take advantage of the case where both processes share physical memory.

If you look up shared memory for Intel MPI, it's confusing since you will see references to controlling message passing in shared memory.  That more traiditional MPI method might also be applicable in your case.
 

0 Kudos
James_T_Intel
Moderator
467 Views

When you say you were using shared memory before using mpiexec, what do you mean by that?  Were you directly accessing shared memory yourself?  And when you started using mpiexec, what modifications did you make to your code?

0 Kudos
Lei_T_
Beginner
467 Views

I mean that when I run engine.exe in GUI, both engine and GUI are able to access the shared memory. But when I run mpiexec.exe -n 4 engine.exe in GUI, the above two processes cannot talk to each other through shared memory. It seems that now engine.exe has its own shared memory among 4 CPUs.

0 Kudos
James_T_Intel
Moderator
467 Views

Without knowing more details of how you are accessing the shared memory, it's difficult to say what the problem really is.  It could be that you have a race in engine such that having 4 instances accessing the shared memory simultaneously causes a deadlock.  What MPI calls are you using in engine?  You might have something else going on there.

Do you have Intel® Trace Analyzer and Collector installed?  If so, you could try running with -check_mpi, which will link to the correctness checking library.  This will check your MPI calls for correctness.  This might provide some additional information.

0 Kudos
James_T_Intel
Moderator
467 Views

I got the code you sent.  It looks like you are introducing a race on the shared memory.  When you run mpiexec with a non-MPI program, there is nothing between the ranks synchronizing them.  They are totally independent programs.  However, your program is accessing shared memory.  With one reader and one writer, everything is fine.  However, when there are multiple writers, all of them are attempting to write to the same shared memory.  This is leading to a race, which locks your program.

0 Kudos
Reply