Multithreaded File I/O

[cfann61]

I have a FORTRAN DLL that is being called by a C++ EXE. The exe creates multiple threads and calls the FORTRAN dll in each new thread. I'm having trouble reading to and writing from files. I am encoutnering one of two errors every time, "end-of-file" or "attempting to write to a READONLY file", which I assume means the file is already open.

The file name should be different for each thread. I wrote a sample with a different file name for each thread.

Where am I going wrong?

subroutine FTHREAD(dcalc,tID)

implicit double precision(a-z)

!DEC$ ATTRIBUTES DLLEXPORT :: FTHREAD

integer uin,uout,tID

character*256 ain,aout

character char

integer int

real float,array(8)

namelist /test/ char,int,float,array

if(tID.eq.0)then

ain='assump.dat'

aout='assump.out'

uin=10

uout=11

else

ain='assump2.dat'

aout='assump2.out'

uin=12

uout=13

endif

open(unit=uin,file=ain,status='old',form='formatted')

read(uin,test,ERR=999)

write(*,*) "FThread: tID = ",tID

write(*,*) "FThread: Assumption Read Complete."

close(uin)

open(unit=uout,file=aout,status='unknown')

write(uout,test)

close(uout)

return

999 write(*,*) "FThread: Error reading assumptions file."

return

end subroutine

[Steven_L_Intel1]

It looks as if you are using the same unit number in each thread. The unit numbers are global to the execution, not per-thread. You are a good candidate for the NEWUNIT= keyword we added to the compiler in Composer XE 2011 - this is a Fortran 2008 feature. You do:

OPEN (NEWUNIT=iun,....)
and an unused unit is selected and returned to you in variable iun after the file is opened. Then make sure you always use the variable in your I/O statements.

Also be sure to select the property Fortran > Code Generation > Generate Reentrant Code > Threaded. This will inform the I/O library that it may be called from a threaded environment.

[cfann61]

I've set up the unit number and file names to be accessed in an array, depending on the thread ID, tID (code below). I set Fortran>Code Generation>Generate Reentrant Code> Yes. I did not have a threaded option, it was Y/N.

It does not work correctly every time. I'm still getting some crossover or something between the threads. I'm not sure exactly what it is though. It seems to me like the data from arrays 'dcalc' and 'array' are being overwritten by the second thread that executes. 'dcalc' is a REAL array that I added. The data in 'array' is read in from the assumptions files.

Code:

data uin/10,15/

data uout/11,16/

data ain/'assump.dat','test.dat'/

data aout/'assump.out','test.out'/

i=uin(tID+1)

j=uout(tID+1)

open(unit=i,file=ain(tID+1),status='old',form='formatted')

open(unit=j,file=aout(tID+1),status='unknown')

[Steven_L_Intel1]

It looks as if you are opening the same file in multiple threads. That is very likely to create problems.

[cfann61]

Maybe I don't understand how the thread works. I've set up the arrays so that each file name will be accessed by thread ID. So thread ID=1 would access 'assump.dat' and 'assump.out'. Thread ID=2 would access 'test.dat' and 'test.out'.

How would the different threads be opening the same file?

I was assuming each thread would have it's own ID, and the file names could be accessed according to the thread ID. Stepping through the debug, it seems like it is accessing the same file, but I don't understand why.

[Steven_L_Intel1]

You don't show how you determine the thread ID. As long as each thread is using its own unit and file, it should work.

[cfann61]

The thread ID is part of the call to the fortran DLL. This is what the dll currently looks like. 'a' is the assumption file name, 'dcalc' is a REAL array, and 'tID' is the thread ID passed from the C++ exe.

In the code below, the problem is writing to the out files. If I open the ".out" file with one of the two commented out open statements, the .dll runs fine. If I use the open statement that is not commented out, the 'b' CHARACTER out file name gets overwritten sometimes, and it trys to write to the same file twice (different units, same file name).

What is wrong with the code the way it's written? If the .dll is called from two seperate threads, why would the memory for the 'b' variable be shared betweent the two threads? Maybe it isn't, but why else would the variable be overwritten?? (Input files are hardcoded to be different file names)

This is confusing the crap out of me.

subroutine FTHREAD(a,dcalc,tID)

implicit double precision(a-z)

!DEC$ ATTRIBUTES DLLEXPORT :: FTHREAD

integer uin(8),uout(8),tID

integer i,j

character*256 a,b,ain(8),aout(8)

logical opend

real dcalc(1000)

character charac

integer int

real float,array(8)

namelist /test/ charac,int,float,array

data uin/10,15/

data uout/11,16/

data aout/'assump.out','test.out'/

i=uin(tID+1)

j=uout(tID+1)

b=a(1:index(a,"."))//"out"

write(*,*) "FThread: tID = ",tID," Open unit = ",i," File = ",a(1:15)

open(unit=i,file=a,status='old',form='formatted')

! open(unit=j,file=aout(tID+1),status='unknown')

! open(unit=j,file=a(1:index(a,"."))//"out",status='unknown')

open(unit=j,file=b,status='unknown')

read(i,test,ERR=999)

dcalc(5)=array(1)

write(*,*) "FThread: tID = ",tID," array(1) = ",array(1)

write(*,*) "FThread: tID = ",tID

write(*,*) "FThread: tID = ",tID," Assumption Read Complete."

close(i)

write(j,test)

close(j)

dcalc(6)=real(tID)

return

999 write(*,*) "FThread: tID = ",tID," Error reading assumptions file."

return

end subroutine

[Steven_L_Intel1]

Yes, the way you have written it, B could be in static storage. Add the keyword RECURSIVE before the SUBROUTINE keyword and see if it helps. Be aware that any variables that you DATA-initialize will also be static.

[cfann61]

I think that did the trick! Thanks!

So are all variables in static storage vulnerable to being overwritten in a multithreaded application?

Just out of curiousity, is there a way to make sure the 'B' variable isn't in static storage without using the RECURSIVE keyword before SUBROUTINE?

[Steven_L_Intel1]

Yes, any static storage is shared among threads. There's more than just variables - the compiler has its own data structures it creates and if you don't say RECURSIVE, or compile with the /recursive switch, then those data structures will also be static and shared. Any time you are doing multithread programming you should use RECURSIVE for everything.

[cfann61]

I have some common blocks in my dll that are sharing data between the subroutines that are internal to the dll. Does the COMMON statement make those variables STATIC?

If it does, is there a way I can get around that (besides passing all the variables)?

[Steven_L_Intel1]

COMMON is always static. No, there is no alternative to passing the variables back and forth. Use of a derived type structure that is your single "context" can make this easier.

[cfann61]

I'm having trouble with the COMMON blocks in the DLL. It seems like the COMMON statement makes the memory for that block of variables STATIC. I need a way to make that memory allocation dynamic.

I found the compiler option /Qdyncom. Is that what I need to use? How would I apply that compiler option in Visual Studio? It would need to be applied to quite a few common blocks.

[cfann61]

Ok, just saw your reply after I posted that last comment. So the /Qdyncom compiler option will not allow the memory for the common blocks to belong to the individual threads?

[Steven_L_Intel1]

No, Dynamic Common is shared among all threads.

In OpenMP, you can name COMMON blocks in a "threadprivate" directive, but this doesn't work in DLLs in Windows (a Windows limitation) and it would depend on the C application using OpenMP rather than its own threading.

[onkelhotte]

I think this code was posted here sometime before, it is a function that determines a free unit:

[bash]! **********************************************************************
integer(kind=4) function getFreeUnit()
! **********************************************************************
  
logical(kind=4) test
  
do getFreeUnit=10,999
    inquire(getFreeUnit,opened=test)   
    if (.not.test) return  
enddo

getFreeUnit=-1
return
end function getFreeUnit[/bash]

Markus

[Steven_L_Intel1]

You don't need that anymore:

OPEN (NEWUNIT=iun, ...)
WRITE (iun....)

[onkelhotte]

I do, my boss wont buy me a new license :-/