open file using newunit directive, threadsafe fortran

[William_G_]

HI Lorri, others,

I have just one more question before I dive in and try to play

with rewriting parts of my application for threadsafe, and it

relates to reading the input data.  

Each thread independently processes in the same way , without any sharing

of results between the threads, data from the single data file submitted to the thread.

The file is identified by a full path, such as c:\mydatafiles\exmpl001.dat as the first argument from the c== calling interface

The results of the processing at the end of each thread are passed back out to the interface

to be reported there, and the thread is closed. In many instances the results are just written back to the interface

in memory. In some applications the results are written out to a physical data file, but first I will be trying how things work

just with the memory dump, to reduce the file handling needed with the writing. 

Each file has the same binary file format, but different data.  Since some of the files could have a lot more data than others

the processing time and usage of features in the subroutines will be different between the threads.

Would this general approach be expected to work ----:???

The only file reading is done in the main routine using

open(newunit=NWI8,file=DSNX,form='binary') 

up front as the first task as soon as the data path and file name are parsed

such as DSNX = C:\mydata\exmpl001.dat,  then do all the file reading in the main routine into arrays,

close the unit NWI8, and then pass the arrays via argument lists to the subroutines downstream.

When I am doing this, I would expect that a valid & different value of NWI8 would be generated for each thread,

so the correct data file would be read in each thread without racing. Should I expect that this would work????

OR,

Should I expect that I have to take more precautions to manage the overall process, such as

I have to have the c++ calling interface that wants to do multithreading to my main program rewritten provide a different

valid unit number to me --  currently the c++ calling interface is only passing the path of the datafile,

and I have no information for how many total threads (meaning individual datafiles that might be wanting to run concurrently).  

Today all I have from the c++ interface is stdcall  myprogram(datafilepath, ......other processing arguments)

whereas I might have to have the calling interface rewritten such as stdcall myprogram(unit number,datafilepath,max number of

threads, .......other processing arguments) 

Finally,  am I correct in assuming that it is better to do all the file reading in the main routine, even though it would be

better & more elegant if I could use a variation of the file reading, which currently is its own subroutine???

thanks for your advice and patience with my questions...

Bill

[Steven_L_Intel1]

NEWUNIT should be thread-safe. It isn't clear to me if you have other questions.

[Tobias_Loew]

Hi Steve,

I have the strong impression that NEWUNIT is not thread-safe: I have a FORTRAN library, which reads from files and gets called parallel from C++ code. The C++ code uses native C++ std::thread (NOT OPENMP) and I get reproducible crashes, which come from different threads using the same file-unit, even though the file-unit is acquired by NEWUNIT.

I could solve the issue by using the thread-id as unit.

Maybe, NEWUNIT works with OPENMP-threads, but I cannot confirm this for arbitrary threads.

Tobias

[William_G_]

hi steve,

Thank you. I will give it a try and see how it goes.

bill

[jimdempseyatthecove]

>>The C++ code uses native C++ std::thread (NOT OPENMP) and I get reproducible crashes, which come from different threads using the same file-unit, even though the file-unit is acquired by NEWUNIT.

This may be a case of the Fortran code assuming if the code is multi-threaded that it is multi-threaded via OpenMP. IOW the "thread safe" incorporates

   if(omp_in_parallel()) then

     !$omp critical(newunit_critical)

    ...

Which would not make the code thread-safe using std::thread.

IOW, it would be your responsibility to make the code correct.

FWIW - the thread safety should be implemented via a CAS or similar technique (LOCK; BTS)

Jim Dempsey

[Steve_Lionel]

You do realize you are replying to a seven-year-old thread? 

There have been issues in the past with NEWUNIT and thread-safety, but I haven't heard of any recently.  I'm sure that Intel Support, from which I retired almost six years ago, would like to see a small example that reproduces the problem.

[Tobias_Loew]

I created a Fortran-Dll which used NEWUNIT to open a text file and read all the lines. Then I made a C++ project which called it massively parallel from std::threads - and everything worked fine! Yeah!

After investigating my large project with the errors, I identified another library which messed up the Fortran-RTL. After fixing that, all the above mentioned errors disappeared when using NEWUNIT. Sorry for the trouble. And thanks for all the replies!