function SYSTEM and SYSTEMQQ

[Michal_Kvasnicka]

Question for Intel fortran guys:

Are the fortran functions: SYSTEM and SYSTEMQQ thread-safe?

[Steven_L_Intel1]

Probably, but I'll ask.

[Ron_Green]

I'm kind of curious why you want to write a Fortran wrapper for this parallel job launcher. My preference is to do these things in bash or another scripting language. For example:

[fortran]#!/bin/bash

exec="myprog.exe"
basein="inputfile"
baseout="outputfile"
extin=".in"
extout=".out"

n=100

for (( i=1; i<=$n; i++ ))
do
 command="$exec < $basein$i$extin > $baseout$i$extout "
 echo $command 
 $command &
done[/fortran]

This simple script allows you to change the program name, input base filename, extension, etc. One could add some arrays of text and index into those if you like for the filenames.
The key is the $command &
which runs whatever command you create in the text var command and executes it in background.

You can put this into a file named something like 'launch.sh' and then chmod +x launch.sh and then
./launch.sh

Is this not flexible enough?

ron

[Michal_Kvasnicka]

The main reason is the fact, that I have very efficient mathematical optimization fortran code and I would like to use them to optimize cpu-expensive black-box objective function based on external standalone simulation program.

On the other hand, I can use above mentioned script file and execute them from fortran code by SYSTEM function, without any additional openMP/MPI paralelization.

Frankly, I prefere paralelization on fortran source code level, because script file is not always fully portable.

[Steven_L_Intel1]

The developers tell me that the routines themselves are thread-safe.

[jimdempseyatthecove]

Then you might want to consider

!$omp parallel do sechedule(static, 1)
do i=1,n
if(SYSTEMQQ(CommandLine(i))) then
result(i) = 0
else
result(i) = GETLASTERRORQQ()
endif
end do
!$end parallel do
do i=1,n
if(result(i) .ne. 0) call oops(i)
end do

Jim Dempsey

[Ron_Green]

And a couple of points to keep in mind:

the OMP solution of Jim's: the OMP scheduler will distribute the work inside the loop in parallel it a thread pool. Let's for example say you have a 4 core system. This thread pool for OMP would be 4 threads, and the interations from 1..n will be carved out amongst 4 threads. Each of the 4 will start CommandLine(i) via SYSTEMQQ and block until that command finishes. The result - 4 simultaneous running jobs. In other words, your workload is being done 4 at a time.

This may be exactly what you want if your jobs consume exactly 100% of 1 core. you will maximize your throughput.

But if the jobs stall a lot waiting for disk IO or network IO, you will be leaving a lot of cycles idle due to this latency.

Contrast that to my scripting solution. My scripting solution launches ALL of the jobs to run concurrently in background. Thus, if the jobs stall a lot waiting for disk, another one will get dispatched to the core and take the free cycles.

Depending on the latency inherent in your workload, there is probably an ideal number of these to run simultaneously on your system.

My guess is that the jobs probably crunch the CPU pretty well without latency on disk, etc. In this case, Jim's is an ideal solution.

A note on Steve's comment: Yes, the SYSTEM and SYSTEMQQ routines themselves are thread safe. This does not mean that you code calling these will be thread safe IF you have side effects of the commands, or dependencies on files, etc. between the program runs. Notice the subtle distinction? The SYSTEM call itself is thread safe but you need to make sure your command sets are thread safe.

ron

[Michal_Kvasnicka]

To Ronald W. Green:

Your last sentence is extremely important.

So, the crucial thing is: How to check out if the external program (or may be a more or less complex script file) is thread-safe or not at all? Is there anystraightforward and simpletest?

[Ron_Green]

For this case, the SYSTEM spawns a new child process. This makes it pretty safe, because the new process will NOT share the same memory space as the parent program and threads. So you don't have to worry about memory races and contention.

For your case, IF the program you start uses distinct files (no file sharing between programs) you should be thread safe. Where it would be a problem is if the programs shared a file and they were updating and reading the same file.

I expect you have a parametric study with a large set of disjoint (unique) input files and each run dumps it's output to a unique output file. In this scenario you are thread safe.

ron

[jimdempseyatthecove · Accepted solution]

You can also change schedule(static,1)to schedule(dynamic,1)
With this change, if the commands (external process) compute for different time intervals, then the next iteration in the list of commands is performed by the next available thread (as opposed to schedule(static,1) being parceled out in a predetermined sequence).

Jim

[Michal_Kvasnicka]

Dear Jim,

thank you very much for your help. Your hints regarding OpenMP implementation of the external process parallel loop are very helpful for me.

Best regards,

Michal Kvasnicka

[jimdempseyatthecove]

Michal,

FORTRAN can call C functions (or subroutines for void C functions). A better way to handle your problem under Linux (Mac) is (may be) to use the C forkv() (or fork()) to create a seperate process (remembering the pid) then launch the command line from that process. On Windows you would use CreateProcess. The reason being is should these processes be serial but with substantial I/O, then you can have more threads than cores. While you could increase the thread count on OpenMP to exceed the number of available cores (HW threads), this will also affect the number of hardware threads use in real computation by the parent process (which may not be what you want).

To be realy generic IVF should consider adding something like

hProcess = PROCESSQQ(CommandLineArgs)
! program continues here while extra process runs
if(hProcess .eq. 0) call oops() ! didn't start for some reason
! do other code here
...
if(.NOT. WAITPROCESSQQ(hProcess, CompletionCode ))call oopsProcess()
! Process done, no error

Note, this would permit multiple concurrent processes to be launched as you can with fork and CreateProcess. The older POSIX may have a function/subroutine to this effect. Since the ...QQ attempts tosupport a generic means for non-FORTRAN functions, something like PROCESSQQ should be considered. Some thought should be given as to the proper name and calling syntax.

Jim Dempsey