Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.
2464 Discussions

Coupling two programs with shared memory

desiramoobranz_co_nz
539 Views
Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs)performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by some linkage.

Thankyou in advance for your help.

Desi
0 Kudos
1 Solution
pvonkaenel
New Contributor III
539 Views

Under Windows I 've used a block of named shared memory to pass data between applications. You use CreateFileMapping() and MapViewOfFile() on one side and OpenFileMapping() and MapViewOfFile() on the other. However you need to be very careful about unsynchronized access into the memory from multiple applications and must use the heavy weight named synchronization objects which are quite expensive.

In my case it work very well in a pipelined system where events were triggered in one app when data was available for consumption in other apps.

Peter

View solution in original post

0 Kudos
6 Replies
jaredkeithwhite
New Contributor I
539 Views

You should look at something like MPI. To my knowledge, modern operating systems are not going to offer you much support in the way of two programs accessing one another's memory. You can find some good resources about MPI here:


Also:


Additionally, the Intel Compiler (for both C++ / Fortran) support OpenMPI.

Regards,
-Jared



Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs)performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by some linkage.

Thankyou in advance for your help.

Desi

0 Kudos
RafSchietekat
Valued Contributor III
539 Views

POSIX support takes the form of shm_open(3) and shm_unlink(3), which build on top of mmap(2).

0 Kudos
pvonkaenel
New Contributor III
540 Views

Under Windows I 've used a block of named shared memory to pass data between applications. You use CreateFileMapping() and MapViewOfFile() on one side and OpenFileMapping() and MapViewOfFile() on the other. However you need to be very careful about unsynchronized access into the memory from multiple applications and must use the heavy weight named synchronization objects which are quite expensive.

In my case it work very well in a pipelined system where events were triggered in one app when data was available for consumption in other apps.

Peter
0 Kudos
adunsmoor
New Contributor I
539 Views
The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI.

IPC: http://www.boost.org/doc/libs/1_39_0/doc/html/interprocess.html

MPI: http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html

I haven't used either of these but happened to read the documentation yesterday. The interprocess library seemed well put together and hid some of the intracacies of using the lower levels interfaces. Especially if you have to do any cross platform work.
0 Kudos
RafSchietekat
Valued Contributor III
539 Views
"The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI."
Going with boost is rarely a bad idea, I was thinking of amending my suggestion with a reference myself.

"Especially if you have to do any cross platform work."
Without boost or even if I don't really need it (all targets would have to provide it), I would probably try to implement shm_open and shm_unlink as adapters in terms of the native API (which was not specified), instead of inventing yet another layer.
0 Kudos
desiramoobranz_co_nz
539 Views
Many thanks for your replies. I have implemented the, CreateShareBuffer(),OpenShareBuffer(),CloseShareBuffer(), DeleteShareBuffer() approach with success. Problem solved now.

Cheers

Desi
0 Kudos
Reply