run parallel (read a file with fixed filename in every run)

[Mike896]

Hi,

I had an fortran-compiled execution file, which will outputs a file with a fixed filename.
I'd like to run this exe file with a loop and read the output file in every run.
In fact, the output file contains different data in every run.
Could it be possible to run it in parallel? Is it conceptually wrong, since read a fixed filename concurrently?
Thanks in advance.

Mike

[jimdempseyatthecove]

Mike,

Your description can be read multiple ways. We need additional information.

1) "I had an fortran-compiled execution file" does this mean you do not have the source code to the file?
2) "I'd like to run this exe file with a loop" if you do not have the source, does this mean run by way of a scripting language such as BATCH?

If you do not have the source, then consider running multiple copies of theapplication in different folders (different directories). Or running the same copy of the applicaiton but with each instance running is a different folder. Giving each a different starting file, then they should run independently (as long as the program uses current directory for input/output).

If you do have the source, and if running multiple applicaitons in different folders is not the design you want, then you have an additional option of coding using multiple threads. If the application has sufficient computaiton between read in and write out then you can (may be able to) perform the computation in parallel.

! serial section of code
call ReadAllOfFileAsRecordsIntoArray()
!$OMP PARALLEL DO
DO I=1,nRecordsRead
! parallel section, each thread receiving different sets of I's
call DoWorkOnRecordInArray(I)
END DO
!$OMP END PARALLEL DO
! serial section
call WriteModifiedRecords()

The above is simplified and may or may not work with your requirements.

Jim Dempsey

[Mike896]

Hi, Jim

Thank you very much for your help. Your information is very precious for me.


I do have the source file of the "fortran-compiled execution file". I create another file which reads input file and run that exe file using batch. I run it with different input parameters and output the file with fixed filename. And, I want to do this many thousand times.

Since I am a newbie in parallel computing and also that source file contains over 10K lines of statements, I don't like to choose the suggested way (I don't understand !$OMP PARALLEL DO) currently.
I don't like "running multiple copies of the application in different folders (different directories".
I prefer "running the same copy of the applicaiton but with each instance running is a different folder".
in ?
My PC contains Quad core, how many folders do I need to create? I guess it's three.
Then I guess I'll create these different foldersand put the batch files in them and run them concurrently.
Am I right? But how to run them concurrently?
Then I read those data and put them into the array.

I don't understand "Giving each a different starting file, then they should run independently (as long as the program uses current directory for input/output)"; doesit mean different batch files in different folder but use the same exe file?

Thank you very much again for your reply.

Mike

[jimdempseyatthecove]

>>I run it with different input parameters and output the file with fixed filename.
Is the input file name and the output file name the same file? IOW are you replacing the file with each run?

>>And, I want to do this many thousand times.
Are these different parameters used on the original file used for input?
Or is each output file (modified by the program) used for the next run with new set of input parameters?

Does your "many thousand times" terminate if the run determines you reached a solution?
Does each run, with different input parameters, run for approximately the same amount of time?
How long is each run time?

The answers to these questons, and perhapse more questions, are important in designing your program flow.

Example:

Each run uses the same input file but with different parameters. Each execution writes to a fixed file name (in current directory) but also returns an exit code indicating "solution found" (or conversely "solution not found").

In this case you could take your"many thousand times" input parameter list and split it four ways. Then run four seperate batch streams. When (if) termination condition is found in any one of the runs, your batch file could conditionally issue ECHO "found">..\found.txt
all batch files use IF EXISTS "..\found.txt" GOTO DONE
In there loop or in between each program run.

--------------
If the output from a run with one set ofparameters is used for input with the next set of parameters, then you cannot seperate the runs into different streams.... Unless you have multiple runs starting with the same input file (contents) but with different starting input parameters.

I cannot suggest a plan of action until you disclose a complete description of the problem.

Jim Dempsey