Coupling two programs with shared memory

[desiramoobranz_co_nz]

Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs)performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by some linkage.

Thankyou in advance for your help.

Desi

[jaredkeithwhite]

You should look at something like MPI. To my knowledge, modern operating systems are not going to offer you much support in the way of two programs accessing one another's memory. You can find some good resources about MPI here:

http://www.mcs.anl.gov/research/projects/mpi/

Also:

http://www.open-mpi.org/

Additionally, the Intel Compiler (for both C++ / Fortran) support OpenMPI.

Regards,

-Jared

Quoting - desiramoobranz.co.nz

Hi,

This is not really a threading problem but I assume the most appropriate place to ask this question. Can anyone offer advice on how to pass data between two independent programs where one feeds data for the other to use as input. I have seen $shared_common statement in HP which puts the common block in shared memory and wondered this existed within intel.

For example I need to do the the following

Program A sets values

Program B takes values (from A using some form of shared memory between the two programs)performs calculations sets new values for input into program A for x time steps.


These two programs are independent of each other and run independently. My research requires they pass data to each other by some linkage.

Thankyou in advance for your help.

Desi

[RafSchietekat]

POSIX support takes the form of shm_open(3) and shm_unlink(3), which build on top of mmap(2).

[pvonkaenel · Accepted solution]

Under Windows I 've used a block of named shared memory to pass data between applications. You use CreateFileMapping() and MapViewOfFile() on one side and OpenFileMapping() and MapViewOfFile() on the other. However you need to be very careful about unsynchronized access into the memory from multiple applications and must use the heavy weight named synchronization objects which are quite expensive.

In my case it work very well in a pipelined system where events were triggered in one app when data was available for consumption in other apps.

Peter

[adunsmoor]

The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI.

IPC: http://www.boost.org/doc/libs/1_39_0/doc/html/interprocess.html

MPI: http://www.boost.org/doc/libs/1_39_0/doc/html/mpi.html

I haven't used either of these but happened to read the documentation yesterday. The interprocess library seemed well put together and hid some of the intracacies of using the lower levels interfaces. Especially if you have to do any cross platform work.

[RafSchietekat]

"The boost library offers C++ interfaces for both interprocess communication through shared memory and MPI."
Going with boost is rarely a bad idea, I was thinking of amending my suggestion with a reference myself.

"Especially if you have to do any cross platform work."
Without boost or even if I don't really need it (all targets would have to provide it), I would probably try to implement shm_open and shm_unlink as adapters in terms of the native API (which was not specified), instead of inventing yet another layer.

[desiramoobranz_co_nz]

Many thanks for your replies. I have implemented the, CreateShareBuffer(),OpenShareBuffer(),CloseShareBuffer(), DeleteShareBuffer() approach with success. Problem solved now.

Cheers

Desi