write/read problems with IVF compiler v. 17

[Thomas_L_]

Updating from ivf compiler version 16 to version 17 (17.0.1.143) has caused some strange behavior when writing/reading from files.

A file is opened written to and read with the following:

      open(unit=17,file=path/name.ext,
     &     access='sequential',form='unformatted',status='unknown', 
     &     action='readwrite')
      write(17) some_large_record (around 36000 4-byte words)
...
      rewind 17
      read(17) some_large_variable 

No problems occur using the compiler version 16. But with version 17 I get the following error when reading from the file:

"forrtl: severe (67): input statement requires too much data, unit 17"

Adding the recl=40000 to the open statement removes the problem, but since this approach is used a lot throughout the code, I would like to know the source of the problem.

Setting recl=30000 gives me a meaningful error when writing to the file ("output statement overflows record") which does not occur when leaving out recl.

[Kevin_D_Intel]

The source of the error is that some_large_record does not equal some_large_variable.

I cannot explain the different behavior between the two compilers off hand. It would be possible with something that demonstrates that. With RECL in the OPEN, you establish the size of each record for direct access or the max size for sequential access. In the absence, that is set based on the size of the list of variables involved in the related I/O.

[mecej4]

The file is opened with FORM='unformatted' and written as an unformatted file, yet you try later to read the file with a format. If this is by design, can you explain how you expect it to work, and why you need to mix formatted and unformatted I/O?

[Thomas_L_]

Both some_large_record and some_large_variable consists of a series of long integer and real arrays. These are verified to have the same size when writing and reading.

As I have understood it, recl= does only specify the maximum record length in combination with access='sequential'

Regarding the format, this is an error in my example, i.e. the code does not contain this format specifier. I have updated the post to reflect this.

[Kevin_D_Intel]

Ok. Perhaps there is a defect in our 17.0 RTL related to this record-length calculation for the read. It would be most expedient getting help analyzing this if you were able to recreate the error with a small complete reproducer. I will make some inquires in the meantime.

You are correct regarding RECL. I tried clarifying my earlier reply.

[Kevin_D_Intel]

Based on the details provided others said the behavior should be the same w/16.0 and 17.0 so it is probable there is a defect at play. If you could help us with a reproducer we would appreciate it.

[Thomas_L_]

Hi Kevin

Thanks for your help so far.

I managed to make an example which behaves like my code. The problem seems to occur due to the first record written to the files. I.e. I observe the following with the example below.

1. Decreasing the real array size from 100000 to 10000 removes the problem
2. Using the ivf compiler in version 16 removes the problem
3. Including recl with a sufficient size in the open statement removes the problem (unit 2)
4. Leaving out the first record (the 4 character string) removes the problem

If something is wrong with the code, please let me know.

program recl_test
      
  implicit none
      
  ! Variables
  integer :: i
  real(8) :: a(100000)
  character(LEN=4) :: c
      
  a(:) = 2.0
      
  open(unit=1,file='C:\TEMP\test1.dat', action='readwrite', &
      access='sequential',form='unformatted',status='unknown')
  
  open(unit=2,file='C:\TEMP\test2.dat', action='readwrite', &
      access='sequential',form='unformatted',status='unknown', recl=200000)
      
  !WRITE
  write(1) 'ABCD'
  write(1) a
  
  write(2) 'ABCD'
  write(2) a
      
  !READ
  rewind 1
  rewind 2

  read(2) c
  read(2) a
  
  read(1) c
  read(1) a
      
  close(1)
  close(2)
    
end program recl_test

[Thomas_L_]

Maybe I should add that writing to the file works well. I have compared two files from the version 16 and 17 compiler respectively. These files are identical. I.e. it is the reading of these files that behaves differently.

[Kevin_D_Intel]

Thank you Thomas for the reproducer. We are still missing/overlooking something here. The testcase behaves for me when built for both IA-32 and Intel64 w/17.0.1.

What compiler options did you use?   What version of Visual Studio are you using?

[mecej4]

I see no run time errors with the code of #7, using the 2017U1 compiler (32-bit or 64-bit) on Windows 10.

One point to note: the unit for the value specified after RECL= is a "file-storage-unit". By default, in Intel Fortran that is four bytes. An -assume option is available to specify the record length in byte units, instead. Other compilers may implement similar variations.

[jimdempseyatthecove]

Try using the 100000 (failing array size), remove the recl..., and adding RECORDTYPE='VARIABLE' to your open statements.

I suspect that without the large recl=... that segmented file format was selected .AND. an error developed in writing/reading a multi-segmented record in your test case (this would be a bug). Using RECORDTYPE='VARIABLE' may resolve this for you.

Jim Dempsey

[Thomas_L_]

I have taken a closer look at my project settings. The setting causing the problem is the buffered I/O.

/assume:buffered_io

I am using the Microsoft Visual studio 2013 shell version 12.0.40629.00 update 5

[Thomas_L_]

Have you been able to reproduce the error yet?

[Kevin_D_Intel · Accepted solution]

Sorry for the delay. We still cannot reproduce it even with that option and your mentioning of it had us wondering if it related to some other reported issues, one in particular is here which I confirmed is fixed in PSXE 2017 Update 2. Without being able reproduce the failure you experienced we could not confirm whether your case was a variant of this or another known case.

If you have your original test case in a project, can you share the build log for that for the case that fails?  Perhaps there is other compiler options being overlooked.

I don’t know whether you want to considering trying PSXE 2017 Update 2 on your end to see whether that may fix the issue or not.

[Thomas_L_]

Hi Kevin

I now got the time for updating to PSXE 2017 update 2. That solved the problem.

Thanks for your help everybody.

[Kevin_D_Intel]

I'm glad to hear that. Perhaps it is a variant of another reported defect then. Please keep us posted should you run into any other related issues moving forward with Update 2.