Calling other programs from Visual Fortran

[gsfantos]

Hi there,

I was wondering if you could have two fortran programs, where the second one is being called durring the run of the first one. Can they interchange information at real time? What commands should I use to start another program in the source file?

Thanks in advance

[Jugoslav_Dujic]

RUNQQ
SYSTEMQQ
CreateProcess
ShellExecute
ShellExecuteEx

As for communication, see this recent thread. It also contains a sample pair of programs communicating using named pipes.

[gsfantos]

Thanks a lot for your very quick answer!!! It's really appreciated.

The problem is that I am not that good in programming and even though I know about the commands you told me I cannot make them working. I haven't really understand if it is possible and how can I do it, from my main program, to call and run another .exe fortran file which will output some results and I will read them from my main. when i am using the runqq command {RUNQQ (filename, commandline)} firstly I don't understand what we put in the commandline.

Thanks again for your help

[Jugoslav_Dujic]

If the program doesn't expect command-line arguments, just supply an empty string ("").

[gsfantos]

Thanks a lot,

I made it working. The problem was that I hadn't understand that even though the exe file you are calling may be even in a different disc, the input file for this program must be in the same folder where the main program is. So I think you helped me a lot.

Now my problem is that actually I don't want the main program to wait until the called one to be finished, but I want the main to continue running and the called one to be active and waiting new input data from the main. Is there any function like runqq that doesnot hold the main program , and do you know if there is any function that delays a program for a specific duration, until the called one has produced the results the main needs, and vice versa

Thanks again for your valuable help, is really appreciated

[Jugoslav_Dujic]

Working directory of the spawned process is the same as the calling process' one (usually, its own directory). You can use CHANGEDIRQQ/CHANGEDRIVEQQ to change the directory to the file's directory.

Now my problem is that actually I don't want the main program to wait until the called one to be finished, but I want the main to continue running and the called one to be active and waiting new input data from the main.

How do you exchange data? Through files? But even then, how do you expect to know when the called program have finished so that you can read the data?

You can use (untested):

SYSTEMQQ("start foo.exe")
CALL SLEEPQQ(nMilliseconds)

However, if you want fine control, you have to use some much finer inter-process communication than simple "wait for a while, it probably completed". Pipes.zip sample from the thread I referred to above does that, pretty much, but don't quite fit into your statement that you're "not that good in programming" :-).

[gsfantos]

Well, the problem is that programming is not my main job, and I am using it just to give "life" to my main job which is maths. The pipes link you sent me it seems very complecated to me. However i was thinking to interchange data through files and using something that good understand if the file is ready to be read to read it, if not wait for another minute and try again. something like this. time consuption is not a problem for now. I am thinking something about the "direct access" reading and the ENDFILE statement, but I haven't about it yet. Worst case scennario I will made them in such a way thateach one must finish first and the continue.

However, I was also wondering if in the case where I have a program waiting for something, I could allocate the memory that is using to a hard drive and free my ram. Do you have any suggestion for this

Thanks again for your help

[Jugoslav_Dujic]

Well, the simplest IPC (inter-process communication) I can think of are named event objects. You can regard it as a "system-wide" logical variable, which you can wait for to become "signalled" (.true.). Here's a sample pair of programs.

Sorry, can't get much simpler than that :-).

As for memory management, don't even try it the system will optimize memory resources better than you can do it.

[gsfantos]

That's great. Thanks a lot , very much appreciated.

Let me have a good look to the files and see how can I attached them to my problem

[gsfantos]

That works perfect. Thanks a lot.

Sorry for being a problem to you, but if you want now to add more "events2" programs you just have to go and create non-signaled events as for event2 but with different names hEvent3.....= (i.e. CreateEvent(NULL, TRUE, FALSE, "My3....."C)) and then work as before individually for each hEvent2....? Also inside the EVENTS3.... you have to specify hEvent= OpenEvent(EVENT_ALL_ACCESS, FALSE, "My3......."C) and the rest are same?

I did and I think is working, but your experience should tell me if I have any mistake ........

Thanks again for your valuable help

[Jugoslav_Dujic]

Yes, that approach should be fine.

Side note: in case you wondered, I inserted small SLEEPQQ's in the sample to ensure that the code won't "roll over", i.e. that the other process will wake up from its WaitForSingleObject and "this" proces will enter its own WaitForSingleObject. If both codes were in two threads the same one process, it is guaranteed that SetEvent will "wake up" other waiting threads immediately (i.e. yield control to other threads). That is not the case with different proceses, so some small waiting to ensure proper synchronization is called for. 100 ms I used is probably an overkill; maybe even zero would do, as it merely instructs the multi-tasker to switch execution to other threads/processes.

[gsfantos]

The SYSTEMQQ is fine, I understand about it.

I did what I said in my last post for more than one separate events. The problem is that when I am asking its individual code to print the hevent number , just to see what is transferred, only for the firsteventthe number is the same with the created by the main program event number, however for the others the hevent number is a number not from the ones that are created, but another one that is also the same exactly for both of them (in case of three sub-events). Do you have any clue why this happens?

The other thing is that I must always pass the control to each one after the previous has finished, or can I pass the control to all sub-events at the same time and wait until again all of them have finished. I tried also this but I didn't manage to get it right. It's not amy main concern, but just to know if I could have them working in parallel

Thanks again,your help is much appreciated

[Jugoslav_Dujic]

Re event handle values: that's normal, and that's just the concept of handle. When you access (open/create) any object (event, file, process, printer, etc.), Windows gives you your own local handle, which is meaningful only in "your" process. Compare: you may OPEN the same file in same or different programs simultaneously, but uner unit numbers 11 and 42 however, both still refer to the same file.

Apart from WaitForSingleObject, there's also WaitForMultipleObjects. There, you can wait for an array of event handles, and tell it whether to wait for all or just for any:

integer(HANDLE):: hEvent(3)

hEvent(1) = CreateEvent(...)

hEvent(2) = CreateEvent(...)

hEvent(3) = CreateEvent(...)

...

!TRUE means that you wait until all become signalled

iRet = WaitForMultipleObjects(3, LOC(hEvent), TRUE, INFINITE)

If 3rd argument is TRUE (wait for all), and all events become signalled, the function will return, and iRet will be WAIT_OBJECT_0 (=0). If it's FALSE, iRet will be index-minus-one of hEvent that became signalled (e.g. 1 means hEvent(2)). If the timeout expired, the return value will be WAIT_TIMEOUT (=258); of course, it can't happen with INFINITE.

Message Edited by JugoslavDujic on 02-14-2006 11:15 AM

[gsfantos]

The waitformultipleobjects works fine for the main program. The problem is that for each individual program once they give the control to the main, they wait the systemqq predefined time and then they go off course to their waitforsingleobject. Since now the main hasn't reset the hevent values because is waiting all the sub-programs to finish, the sub-programs one by one as they finish pass their waitforsingleobject and continue.

So I suppose there must be a way for the sub-programs not to overflow like this. What am I not doing right or what am I missing?

[Jugoslav_Dujic]

Sorry, but I don't follow what exactly you're trying to achieve. Of course, the "sub-programs" don't have to call WaitForSingleObject if they don't have to wait.

Reading between lines, I assume you want some kind of "central waiting" scheme. Note that you may use two hEvents for communicating between the main program and the sub-program, one for each "speaking" direction. Also, you may use "multiple-wait" scheme in the following manner:

!In main program:
integer(HANDLE): hEventListen(3), hEventTalk(3)
...
do while (.true.)
  iRet = WaitForMultipleObjects(3, LOC(hEventListen), FALSE, INFINITE)
  select case(iRet)
  case(0)
      !1st program has finished something:
      READ(...) some results
      !Now, "say" to program 1 that it may proceed. It should
      !ResetEvent(hEventListen(1)) in turn, to enable waiting.
      SetEvent(hEventTalk(1))
  case(1)
      ...
  end select
end do

I admit I didn't foresee in the start it would get complicated like this :-).

At this point, you might consider using (named) mutex objects instead of events. They're fairly similar, except that they "know" who is their current owner, i.e. WaitFor... will not halt if that same thread/program owns the mutex. A mutex can be "released" (ownership granted to others) by ReleaseMutex, and will become owned after a WaitFor.. it returns. Note, however, that it can become "abandoned" if a thread/process dies while it still owns it don't forget to CloseHandle of it (actually, you should always CloseHandle for every created/opened object).

I'm not saying this is necessarily a way to go, but it may become slightly simpler sync issues can get complicated, and may even require a Gantt chart to be gotten right.

[gsfantos]

Thanks a lot for everything. I managed to do it before reading your last post. However I will have a closer look to the mutex scheme.

Basically after each subprogram was settingeach hevent (made it signalled) I placed a waitfor multipleobject command, in order to wait until all the sub-programs had finished. After this command I placed a sleepqq(1000) so when all the hevent were signalled, each sub-program was waiting 1000 before to go to waitsingleobject, and at the same time the main was reseting the hevents (I assume that the 1000 predefined will be enough for the main to loop over and reset the hevents). After the 1000 for each sub-program, the hevents were not off course signalled and therefore all the sub-programs now were waiting the main to signal all their hevents.

In that way I manage not to have an overflow for the subprograms.

Thanks again for everything, without you it seemed to me now something impossible to be done by me. Thanks a lot

[gsfantos]

Hi there again,

do you know if there is a limit on how many parallel programs I can run in win32? I went up to 64 but I cannot make it more than that. Even though I have specified: iret=WaitForMultipleObjects(MRVE,LOC(hEvent),TRUE,INFINITE)

they don't wait and they start run. That happens when mrve is more than 64.

Any ideas?

[Jugoslav_Dujic]

A quick Googling confirms that that's the case – WFMO has a limit of 64 wait handles. I didn't seek for workarounds though.

[gsfantos]

Thanks a lot for the very quick reply. I am trying to find a way to overcome this-if there is any way to do it. I checked that CreateEvent creates more than 64. only the WFMO waits up to 64.

If I won't find anything, I am thinking of calling the sub-programs from interface programs between my current subprograms and main. Will see....

Thanks again

[Jugoslav_Dujic]

Yep, the problem seems to be only with WFMO -- the docs indeed mention MAXIMUM_WAIT_OBJECTS.

A quick idea off the top of my head: when you get close to the limit (e.g. 60 objects), spawn another thread (CreateThread) and in its ThreadFunc wait for the rest (the ThreadFunc must be a STDCALL integer function with exactly one argument). In the main thread, wait for first 60 processes and the thread handle. In this way, you "simultaneously" wait in two different places.

[gsfantos]

Hi there again!!!,

sorry for being such a headache. I managed to run more than 64 threads. But I experience problems with synchronization and I am thinking to use mutex now instead of events. I am trying to built a test example with muttex but I haven't manage to open the mutex in the sub-program in a proper way. Probably I don't specify it correctly. The create and release is ok.

Any suggestions?

Thanks again in advance