File handling differences between Compaq and Intel compilers

[Chris_H_4]

Hello, me again with my crusty old Compaq VF code. Further to my previous issue with files - https://software.intel.com/en-us/forums/intel-visual-fortran-compiler-for-windows/topic/606997 - I have found that the code I am trying to upgrade from Compaq to Intel seems to be rather lax in opening and closing files. The problem may be compounded by the fact that the project I am working on compiles to well over 100 DLLs and for example with the issue linked above the FORT.1 is opened in one DLL but read in another, on entry to the second DLL the file appears to be closed. I got past that by explicitly opening it in the second DLL (bit of a hack but good enough for now). Then I hit another problem where a FORT.7 file is being written in one DLL without being explicitly opened and then a second DLL attempts to write to it and gets the error:

forrtl: severe (30): / process cannot access file because it is being used by another process

I have tried various things to fix this, e.g. explicitly opening and closing the file, the SHARED option, etc. and so far I have not found an appropriate combination but probably could eventually.

HOWEVER, I feel that if I fix that particular issue I would just find another one so I was wondering if there is an "I don't care about proper file handling" compiler flag that would make IVF behave like CVF did?

I also wonder if combining some or all of the DLLs into one DLL may get past some of these problems but since that is a fairly tedious task I would rather not start unless there was some certainty that it would work. One thing that makes me think it may not is that the DLLs are called by an executable built from PL/I so there will be a call to AAA.DLL which will return to PL/I then a call to BBB.DLL, AAA.DLL may have opened/written the file which is then accessed by BBB.DLL. Combining these into ABIG.DLL will still have two separate invocations of ABIG.DLL and so may suffer the same issues. Thoughts on this would also be appreciated.

[mecej4]

I think that if you stay alerted to how I/O is performed and assess the concurrency/sharing issues, you may be able complete the porting work with much less trouble, without having to resort to relying on the OS to enable file access sharing, etc. I assume a couple of things: 1) Only one EXE or DLL will perform I/O on a file at a given time, and 2) in your project there is only one EXE running that will call your DLLs that perform I/O. With these assumption, the issues boil down to making sure that the EXE and the DLLs all share a single Fortran run-time (RTL). 

To help break the logjam, I created a small test project with one EXE and one DLL, in which the I/O issues as seen by the user are the same whether CVF 6.6C or IFort 16.0U1 is used to build. The DLL reads a text file that exists, and displays the data. The EXE creates the text file, writes some data to it and, with the file still open, calls the DLL to read and display the contents of the file. The DLL itself does not open or close the file.

The source for the EXE:

program xptl
   implicit none
   integer i
!
   open(11,file='dfil.txt',status='replace',mode='readwrite')
   write(11,'(2I8)')(i,i*i,i=10,19)
   call rdfil()
   close(11)

end program

The source code for the DLL:

subroutine rdfil()
!DEC$ ATTRIBUTES DLLEXPORT :: rdfil
   implicit none
   integer i,j
!
   rewind(11)
   do
      read(11,'(2I8)',end=10)i,j
      write(*,'(2I4)')i,j
   end do
10 return

end subroutine

I built the DLL using the command df /LD rd.f90 with CVF6.6C, and then built the EXE using the command df /MD pgm.f90 rd.lib. Running pgm.exe produced normal output. I then recompiled only the EXE  using /MT instead of /MD, with the same rd.lib as before, and the program aborted, with an error message from the DLL that it could not find the file fort.11. This is as I expected, because the EXE and the DLL are now using different RTLs, so the DLL neither knows the name of the file nor that it was opened with unit number 11 in the calling process.

I opened an IFort-32 command window and repeated the test of the preceding paragraph, using ifort instead of df in the commands. Once again, when I used /MD to build the EXE it ran fine, but when I used /MT it could not find the file.

Perhaps you can try this experiment yourself and, if the assumptions that I made and my explanations are acceptable, implement the same ideas in your large project.

[Chris_H_4]

@mecej4  many thanks for taking the time to post that. I have tested your suggestion and get the same results and I am sure what you suggest is worth investigating. However there are a couple of differences; firstly* the EXE is built from PL/I not Fortran so I cannot apply the /MD option and I cannot find an equivalent for the IBM VisualAge PL/I compiler. It does use the standard Microsoft linker though so may be there is a linker setting that is equivalent.

The second difference is that the code finds the file but either it is closed when expected to be open, or is in use when being written. I understand that this could well be related to different run-times or run-times on different threads as your example suggests and I will continue to investigate.

Again thanks for the detailed response.

*please accept my apologies, I should have mentioned this in the original post

[Steven_L_Intel1]

There aren't differences between the compilers here. I would bet a quarter that your CVF project has /fpscomp:general set and your Intel Fortran project does not. This would explain the difference, since one of the effects of /fpscomp:general is to default to write-sharing.

[mecej4]

Chris H wrote:
I cannot apply the /MD option and I cannot find an equivalent for the IBM VisualAge PL/I compiler.

The IBM VAPLI 2.1.10 compiler does provide LIBS(SINGLE DYNAMIC) and LIBS(MULTI DYNAMIC) options, but those are probably not relevant.

If, as I suspect, you do all Fortran I/O (opening, reading writing and closing of files) in DLLs built from Fortran source code, and not in the PL/I EXE, the points mentioned in #2 should provide a solution for you. Even if a text file is written by PL/I, as long as PL/I closes it before a Fortran DLL opens it for reading, or vice versa, things should be OK. What you cannot do is to have a file opened in PL/I and then call a Fortran DLL to do I/O to the file, or open a file in a Fortran DLL and try to do I/O on that open file in PL/I.

Do note that there is sometimes a lag in a CLOSE(unitno) statement being completed and the OS actually closing the file. In such cases a call to a "sleep for xx milliseconds" service routine may need to be used between closing a file in one DLL and trying to reopen it in another DLL.

It has been many years since I used PL/I, but I should still expect that there is no easy means for doing I/O on open files in more than one language. I still have a copy of VAPLI 2.10, and it works (at least for console applications) on Windows 10. I tried it out on the kind of problem that you described as follows.

• PL/I main program opens writes some data and closes a data file (20 integers, one per line).
• PL/I main program (after closing file) calls subroutine in Fortran DLL, which opens, reads and closes the file, then calculates and returns the sum of the read values through the lone subroutine argument.
• PL/I main program prints the sum returned from Fortran DLL.

In 32-bits, using CDECL linkage between PL/I and Fortran, using VAPLI 2.1.10 and IFort 16.0.1, everything went smoothly and the correct result was printed, giving the green signal to try more complicated mixed language processing with PL/I and Fortran. Main program source:

  /* Test program to print numbers to a file, call Fortran DLL to read file
   and return sum of values read.
   */

 pgm: proc options(main);

   dcl tmpf file stream output;
   dcl (I,sm) fixed bin(31);
   dcl psum entry(fixed bin(31)) ext('PSUM') options(linkage(cdecl));

   open file(tmpf) output title('/dfil.txt,append(n)');
   DO i=1 TO 20;
    put file(tmpf) edit(I*I)(F(5)) skip;
   end;
   close file(tmpf);

   call psum(sm);

   put edit ('Sum = ', sm) (A(6),F(4)) skip;
 end pgm;

Subroutine used to build DLL:

subroutine psum(ssq)
!DEC$ ATTRIBUTES DLLEXPORT :: PSUM
implicit none
integer isq,ssq
ssq=0
open(11,file='dfil.txt',status='old')
do
  read(11,*,end=10)isq
  ssq=ssq+isq
end do
10 close(11)
return
end subroutine

Building DLL and EXE:

s:\PLI> ifort /LD psum.f90
s:\PLI> pli pgm.pli
s:\PLI> ilink pgm.obj psum.lib

[Chris_H_4]

Thanks again for the responses.

Steve-  /fpscomp:general made little difference but did put me on the trail of tracking down the correct set of options. So breathed life into an old XP version of CVF and went through the compiler settings, matching them up as best as I could. Here's what I ended up with which allows the code to run a good way through the test case I have

/d_lines /Qsave /names:uppercase /module:"Debug/" /object:"Debug/" /Fd"Debug\vc120.pdb" /traceback /libs:dll /threads /dbglibs /c 

I'm not out of the woods yet but it does seem to run without complaining which is a big step on. Oddly I think it's the "/names:uppercase" that made the difference.

mecej4 - thanks for taking the time but as you can see from the above I believe have found a suitable set of compiler options.

[mecej4]

Well, /names:uppercase is turned on by default on Windows, so explicitly specifying it should not change anything unless you have other options that have changed the default.

[Steven_L_Intel1]

/Qsave is more likely the difference, indicating you have a coding error that assumes local variables are SAVEd by default. I agree that /names is not relevant (it would cause linking errors if different).

[Chris_H_4]

Again, thanks for the replies, I believe I had \Qsave on but with over a hundred projects it is possible that one or two got  missed. Anyway I am over that hump and struggling with the next bit of weirdness so I may be back with another thread...

I can see you . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  heading for the hills  :-)