Communication between exe-files

[ijenny]

Hi !

I am quite new at this, so be gentle :)

I have multiple exe-files to include in one program. This makes the communication a really hard part - right now I am communicating with files, but it feels like a bad solution. What I want to do, is start the exe-files from my 'main module', read their results and plot them. But I dont want the main program to be locked while the other programs is calculating. In some cases I want to send 'plotcommands' to the main exe-file but without ending the other exe-file.

Kind of a messyexplanation, hope you understand ...

So, what is the best solution?

[Jugoslav_Dujic]

Before diving into details, few questions...

1) Is the "main module" supposed to be a GUI application?
2) Do you have the sources of "submodules". If you do, can they be modified? If so, is it (relatively) easy?

Take a look at:

http://www.xeffort.com/xeffort/samples/xscicalc/xscicalc.htm

Does it resemble what you want to do?

[ijenny]

Thank you for answering !

Well, yes, the problem is the instruction is to keep all the modules as seperate exe-files. But I can make changes inside them (I wrote them to begin with). And yes, the main module is a GUI application, and the other modules also havea dialogbox shell. And I want changes made in thesecond module to reflect in the image produced by the first exe-file.

The program in the link seems to be about what I am trying to do. But with multiple exe-files. So, the answer to your questions :

1. Yes!

2. Yes ! (but they have to remain separate exe-files)

[Jugoslav_Dujic]

OK then. There are various methods of inter-process communication (IPC) under Windows, but I'm going to propose the simplest (if not the most efficient) pipes.

Pipes are kind of "interactive" files that can be used for interprocess communication. You can also redirect a program's standard output to a pipe (like to a file, e.g. "program.exe > output.txt"), and the calling process can READ from a pipe using pretty standard READ. That would require little or no changes to the "submodule" codes also. There are other ways for IPC (like mailslots, WM_COPYDATA, MapViewOfFile) but pipes are pretty simple, and the concept also works in Unix (if you ever want to port it)

Start with this MSDN article. In the "main module" you should CreatePipe and set up the standard input and output handles for the spawned process, then CreateProcess.

In the "submodule", make sure you read/write from/to standard input/output (READ(*,)) rather than from/to an OPENed file (you can always make the submodule read/write from/to files by redirecting I/O to file that's how most Unix tools do).

Also, note that you can use Fortran READ/WRITE instead of ReadFile/WriteFile. On the "main module", you must use WriteFile/ReadFile, but also take a look at USEROPEN= option in OPEN statement it allows you to OPEN a "file" (unit) using CreatePipe, so that you can use subsequent READ and WRITE to/from the pipe.

Most simply, use ASCII (formatted) read/write take care about line/record terminators (char(13)//char(10)) if necessary. It's not so efficient as binary (you can use that instead), but it would be easier to develop & debug.

As for the GUI front-end, you can (re)use Xeffort code (XSciCalc sample has embedded "calculation" within, but you can replace it with pipe read/writes) or whatever tool you're familiar with. (XSciCalc also demonstrates some important concepts on synchronization, i.e. how you wait for the other task to finish).

I suggest that you start with the MSDN sample, translate it to Fortran, play with READ/WRITE/USEROPEN and simplify it, then apply the concepts to your real task. I can help you with general problems, but I didn't do the same thing from the start to the end myself.

Hope this helps,

[ijenny]

Oh, thank you very much for your answer! You are so sweet!

I have been thinking about pipes, but I have never used them. The only question is - how does the main module know when the other module has written something in the pipe? Does it get som kind of 'you've got mail' message? Or should I do a timerloop checking if something new is written? And if that's the case, cant I just use ordinary files instead?

But I will check it out immediately, really nice to be pushed in the right direction :)

[Jugoslav_Dujic]

I don't really know :-). I thought you didn't want files at all.

Now, I see I omitted an important piece of information; if I rephrase your original question to read:

How do I spawn another process from a GUI application and wait for it to finish (without blocking the GUI)?

Then, the answer is (somewhat hidden) in XSciCalc sample, and unrelated with file versus pipe versus another type of IPC. In the sample, XProcessMessages function is used (for spawning Notepad and for other things), which is built around MsgWaitForMultipleObjects API. Beneath the complicated wording, that function basically checks whether a handle is signalled (e.g. Notepad process has finished) or a UI message arrived. If the former, it means that the process is done. If the latter, it means you should do something in GUI (process the message).

I attached the source of XProcessMessages (you can see the usage docs in .chm on Xeffort download page) it's somewhat overgeneral for your needs though (and not compilable as-is, but I think you should only remove references to xgApp and USE DFWIN).

Note that it's unrelated with file-vs-pipe issue; for example, pipes will let you draw the graph interactively as the pipe is filled in by the child process, but a plain file will serve as well if you're content with drawing when the submodule is finished.

[ijenny]

Well, I don't want to usefiles because they are instable, sometimes the file hasn't been closed before the other process tries to read it and so on ... So pipes are a better approach.

But the pipes doesn't send any kind of interrupt to the main process, telling it that something new has been written there? So, if I do use pipes, I have to check messages from it with a certain timestep. Which of course is a perfectly good solution (if I can get it right, that is ;) ). I will try ! Thank you so much for all help!

[Jugoslav_Dujic]

Try and see what pipes exactly do when the pipe is empty (EOF reached when reading the pipe).

If it doesn't work as you expect, I you can use named events or mutexes to "ping" the other process to tell it that there's something in the pipe that can be read, e.g:

!main module:
hMutex = CreateMutex(NULL, FALSE, "MyMutex"C)
CreatePipe(...
CreateProcess("submodule.exe"...

bPipeNotEmpty = .TRUE.

DO WHILE(bPipeNotEmpty)
   iRet = WaitForSingleObject(hMutex, INFINITE)
   bPipeNotEmpty = ReadFile(hPipe...).NE.FALSE
END DO

!submodule
!We own the mutex now
hMutex = CreateMutex(NULL, TRUE, "MyMutex"C)
...
DO WHILE (SomethingToDo)
   !Wait until main module reads the output:
   iRet = WaitForSingleObject(hMutex, INFINITE)
   WRITE(*,*) results
   ...
   ret = ReleaseMutex(hMutex)
END DO

...or something along these lines :-). I used WaitForSingleObject above (blocking the GUI thread) but you should use MsgWaitForMultipleObjects instead to enable message processing during the data exchange.

I must say you got me intrigued with the problem. I'll try to scrap up a sample.

Message Edited by JugoslavDujic on 02-07-2006 09:23 AM

[Jugoslav_Dujic]

OK, it took some time... I got a handle mixup, but I sorted it out eventually. It's attached.

The good news is that pipe read/write is synchronous, i.e. READ/ReadFile will wait until something comes into the queue. That means that you don't need any external synchronization mechanisms. The other good news is that you don't need ReadFile/WriteFile you can redirect your own stdin/stdout to appropriate pipe ends. In this way, you lose the ability to READ(*) and WRITE(*) into main module's console, but that's not necessary in a GUI app anyway.

However, the blocking nature of READ also means that the GUI will be stalled while READ is being executed. You should assure that you either READ as soon as something is written, or to read data in larger chunks. If the amount of data is large and the transfer takes a long time, consider calling an "asynchronous message loop" (equivalent of XProcessMessages() with no arguments) in between READs:

do while(PeekMessage(mesg, NULL, 0, 0, PM_REMOVE))
   if (.not.DlgIsDlgMessage(dlg)) then
     iret = TranslateMessage(mesg)
     iret = DispatchMessage(mesg)
   end if
end do

[ijenny]

Puh, well, I have tried some of the code now, and I got it to work fine when the main module waits for something to read. However, I couldn't get it to work when I try to send 'print me' to the main module when I push a button in the submodule.

My test is really simple, I open a graphics window in the main module and I try to start a submodule with buttons saying 'draw red line' or 'draw blue line'. Right now the window remains black though :)

I have a bit of a trouble because the application is using GINO Menustudio, so the windowsmessagesloop is kind of hidden. But most of the commands I can find a way to reach. But PeekMessage and similar functions are hard to dig up. This also makes it hard to show some samplecode...

But I think this way might be the solution, I just have to find a way to read immediately when there are something to read in the pipe. I will continue trying!

[Jugoslav_Dujic]

You can use above message loop from anywhere just move it to another subroutine & remove DlgIsDialogMessage (the "universal" version as in XProcessMessage is slightly more complicated, with GetActiveWindow/WS_EX_CONTROLPARENT test and IsDialogMessage). Whenever it's called, it will (at least) refresh the windows' contents correctly.

It won't help you, though, while the READ is being blocked. If you can't make it synchronized in a simple manner, you can resort to IPC via named events (CreateEvent/RaiseEvent) and wait for the event to be raised in another application via MsgWaitForMultipleObjects (as in XProcessMessages); only then READ the pipe.

[ijenny]

But, now I feel kind of stupid, but doesn't the WaitForMultipleObjects also block the program? Or is it something I have misunderstood in the documentation?

[Jugoslav_Dujic]

It does; MsgWaitForMultipleObjects doesn't. See source of XProcessMessages.

[ijenny]

I just want to send the message that it worked fine, and I now send an event telling the main process to read the pipe :) Really great, Thank you so much for all valuable help (and all patience you've had with me ...) !!

The only problem is that the CreateEvent-call gives me a warning about the types?

(hEvent is an integer)

hEvent = CreateEvent(NULL, .TRUE., .TRUE., "MYMESSAGE"//CHAR(0))

gives : Warning: The data type of the actual argument does not match the definition.

[Jugoslav_Dujic]

It should be integer constant TRUE (from DFWINTY) rather than logical .TRUE.. The latter will work in most contexts, but it's better to play it on the safe side (and avoid warnings).

[ijenny]

Thanks :)

It works terrific now !!

[mamey4]

I know this is quite an old thread, but I'm working on a similar task right now and I can't quite get my programs run correctly.
I have three command line programs p0,p1,p2: p0 is a middleware software that starts the other two, p1 writes a value to the standard output, and p2 reads a value from the standard input and saves it in a file. What I want to do is to start p0, which first redirects the standard outputs to some self-created pipes; then p0 starts p1 and p2, which do their writing and reading NOT from the standard I/O, but from the created pipes. Here's an overview:

p0
---

!create 2 pipes
!...
iret = CreatePipe(LOC(hChildStdinRd), LOC(hChildStdinWr), SA, 0)
iret = CreatePipe(LOC(hChildStdoutRd), LOC(hChildStdoutWr), SA, 0)
!...

!redirect I/O
!...
iret = SetStdHandle(STD_OUTPUT_HANDLE, hChildStdInWr)
iret = SetStdHandle(STD_INPUT_HANDLE, hChildStdoutRd)
!...

!start p1 and p2
SI1.cb = sizeof(SI1);
SI1.hStdError = hChildStdoutWr
SI1.hStdOutput = hChildStdoutWr
SI1.hStdInput = hChildStdinRd
SI1.dwFlags = STARTF_USESTDHANDLES
PI1 = T_PROCESS_INFORMATION(0, 0, 0, 0)
iret = CreateProcess(NULL, "p1.exe"C, NULL, NULL, TRUE, 0, NULL, NULL, SI1, PI1)
!analogous for p2, with SI2 and PI2

!wait until p1 and p2 are finished
iret = GetExitCodeProcess(PI1.hProcess,loc(status))
do while( status == STILL_ACTIVE )
iret = GetExitCodeProcess(PI1.hProcess,loc(status))
end do
write(0,*) 'p1 finished!'
!analogous for p2, with SI2 and PI2

!tidy up code, close handels etc
!...


p1
---
PROGRAM p1
write(*,*) 111
END PROGRAM p1


p2
---
PROGRAM p2
IMPLICIT NONE
INTEGER :: i,iErr

READ(*,*,IOSTAT=iErr) i

OPEN(22, FILE="d:/p2.txt")
WRITE(22,*) "i=", i
CLOSE(22)

END PROGRAM p2


If I do it this way the read-command in p2 blocks the composite program, meaning that the value written by p1 never arrives there. If I write something from p0, however, the read in p2 does not block. On the other hand, the write from p1 can be read by p0. So this tells me that the communcations p0-p1 and p0-p2 seem to work, while there are problems for p1-p2.
Do you have an idea what could go wrong?

mamey.