How to let Intel Fortan Project A wait for Intel Fortran Project B before the next circle

[Gao__Wu]

I have two Intel Fortran Projects A and B running parallel on Visual Studio 2010, Windows 10. The goal is to achieve the data transmission from Project A to Project B for each time step.

The logic is like this: at the current time step N, Project A creates the data transmission file “dataintoB.dat”. From the side of Project B, it opens and reads “dataintoB.dat”. Meanwhile, Project A should wait for Project B to finish opening and reading “dataintoB.dat” before proceeding to the next time step N+1.

To notice Project A that the opening and reading process by Project B is complete and continue running, I used the following method, which includes creating a “dataintoA.dat” file by Project B:

open(1, file = 'dataintoA.dat', form = 'formatted')    % created by Project B 
   write(1,*) 1    % a random number is used
close(1) 

and checking the existence of the file by Project A:

INQUIRE (FILE = "dataintoA.dat", EXIST = exists)   
if (.NOT. exists) then
    call sleep(30)     % assume 30 seconds is needed
end if 

However, the above method is not efficient considering the 30 seconds sleep time in Project A for each time step. Is there any other method to improve the efficiency of this operation, such as a 'wait' instruction for Project A before it steps into the next time step? Plus, the Intel Fortran Projects A and B are independent of each other, and both contain thousands of lines of codes and I would not like to change them into one program. Thanks in advance.

[Arjen_Markus]

Well, a simple method is to do something along these lines:

do
    INQUIRE (FILE = "dataintoA.dat", EXIST = exists)   
    if (.NOT. exists) then
         call sleep(1)     % assume a granularity of 1 second is useful
     else
         exit
     end if 
enddo

 There are other more sophisticated methods  possible, like sending messages between the processes, but this is easily built in and the advantage is that you waste at most a single second (or whatever wait time you specify).

[Gao__Wu]

Thanks for your reply.  The current method still needs to specify a wait time however it depends on the performance of Project B and is uncertain.

I am very interested in the method of sending messages between the processes, can you provide more information on how to achieve it? Thank you.

[Arjen_Markus]

Well, the do-loop I suggest means that program A waits for a short while to check on program B. That while may be as short as you want. If program B requires five seconds for the completion of its task, then program A will wait five seconds, but if it needs only one second, program A only needs to wait one second. So in a sense it is dynamic and almost optimal.

Sending messages between programs requires inter-process communication and there are various ways to achieve this. The details depend on the platform you are on. Some possibilities:

• Use MPI to set up communication between two tasks in one program - programs A and B should be put into the same overall program as subroutines,  but that way the tasks can wait for each other via thevairous MPI routines. The programs (turned into subroutines) need not share data directly.
• With coarrays you can achieve the same thing, but then many of the details are taken care of by the compiler.
• Even with OpenMP you can achieve that :).
• A completely different possibility is to use shared memory - two processes sharing a piece of memory. Complication: yo uwill have to use some C functions to set up the memory and store/read data.
• Other mechanisms are available as well, such communicating via sockets, but this all becomes fairly complicated pretty soon.

Hope this gives you a bit of an idea.

[mecej4]

Gao__Wu:

It seems to me that what you really should have done is to have your Project A create an EXE, and have Project B create a DLL. The EXE calls the main subroutine of the DLL to do a specific processing of the data that is shared between the two. It may even be possible that, instead of passing data from the EXE to the DLL through a data file, with synchronization issues, you simply pass that data through  subroutine arguments, shared module variables, etc.

When the two processes are running, the notions of Project A, Project B, etc., have little relevance (the project idea makes sense when compiling and linking the EXE/DLL from source files.) The master process calls the slave process only when it has the input data ready for passing to the slave. Likewise, the slave process returns control to the master process only when it is finished. The operating system takes care of allocating the CPU without wasting CPU resources.

Is there any reason why you cannot switch to this EXE+DLL way of operation, with data passed through memory rather than files?

[Gao__Wu]

Thanks for your reply.

In my case, I think passing data through a file is more straight-forward and is easy to check the values of data.

[Steve_Lionel]

I would use a Windows mutex object for this, if one doesn't want to go the DLL route.

[mecej4]

Intel Moderators:

I noticed something quite odd in the display of Steve_L's response today (02-22-2021 08:53 AM). His post is only one line long, and the body got displayed with scroll buttons in a panel whose height was too small to show that single line. Only by clicking on the scroll buttons was I able to make that line visible.

OneLiner

[jimdempseyatthecove]

do
  SLEEPQQ(100) ! wait 100ms
  INQUIRE (FILE = "dataintoA.dat", EXIST = exists)   
while (.NOT. exists)

That should be relatively low overhead.

Jim Dempsey

[Steve_Lionel]

I'll comment also that the sample DLL_Shared_Data from the Samples Bundle, demonstrates sharing data between two processes and synchronizing that access with a mutex.

[JohnNichols]

I use both methods. 

it is much safer to use the exe and the dll and share the data than write to a dat file and reread.  

It actually makes the programming easier.