Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
1 View

write/read problems with IVF compiler v. 17

Jump to solution

Updating from ivf compiler version 16 to version 17 (17.0.1.143) has caused some strange behavior when writing/reading from files.

A file is opened written to and read with the following:

      open(unit=17,file=path/name.ext,
     &     access='sequential',form='unformatted',status='unknown', 
     &     action='readwrite')
      write(17) some_large_record (around 36000 4-byte words)
...
      rewind 17
      read(17) some_large_variable 

No problems occur using the compiler version 16. But with version 17 I get the following error when reading from the file:

"forrtl: severe (67): input statement requires too much data, unit 17"

Adding the recl=40000 to the open statement removes the problem, but since this approach is used a lot throughout the code, I would like to know the source of the problem.

Setting recl=30000 gives me a meaningful error when writing to the file ("output statement overflows record") which does not occur when leaving out recl.

 

 

0 Kudos

Accepted Solutions
Highlighted
Employee
1 View

Sorry for the delay. We still

Jump to solution

Sorry for the delay. We still cannot reproduce it even with that option and your mentioning of it had us wondering if it related to some other reported issues, one in particular is here which I confirmed is fixed in PSXE 2017 Update 2. Without being able reproduce the failure you experienced we could not confirm whether your case was a variant of this or another known case.

If you have your original test case in a project, can you share the build log for that for the case that fails?  Perhaps there is other compiler options being overlooked.

I don’t know whether you want to considering trying PSXE 2017 Update 2 on your end to see whether that may fix the issue or not.

View solution in original post

0 Kudos
15 Replies
Highlighted
Employee
1 View

The source of the error is

Jump to solution

The source of the error is that some_large_record does not equal some_large_variable.

I cannot explain the different behavior between the two compilers off hand. It would be possible with something that demonstrates that. With RECL in the OPEN, you establish the size of each record for direct access or the max size for sequential access. In the absence, that is set based on the size of the list of variables involved in the related I/O.

0 Kudos
Highlighted
Black Belt
1 View

The file is opened with FORM=

Jump to solution

The file is opened with FORM='unformatted' and written as an unformatted file, yet you try later to read the file with a format. If this is by design, can you explain how you expect it to work, and why you need to mix formatted and unformatted I/O?

0 Kudos
Highlighted
Beginner
1 View

Both some_large_record and

Jump to solution

Both some_large_record and some_large_variable consists of a series of long integer and real arrays. These are verified to have the same size when writing and reading.

As I have understood it, recl= does only specify the maximum record length in combination with access='sequential'

Regarding the format, this is an error in my example, i.e. the code does not contain this format specifier. I have updated the post to reflect this.

0 Kudos
Highlighted
Employee
1 View

Ok. Perhaps there is a defect

Jump to solution

Ok. Perhaps there is a defect in our 17.0 RTL related to this record-length calculation for the read. It would be most expedient getting help analyzing this if you were able to recreate the error with a small complete reproducer. I will make some inquires in the meantime.

You are correct regarding RECL. I tried clarifying my earlier reply.

0 Kudos
Highlighted
Employee
1 View

Based on the details provided

Jump to solution

Based on the details provided others said the behavior should be the same w/16.0 and 17.0 so it is probable there is a defect at play. If you could help us with a reproducer we would appreciate it.

0 Kudos
Highlighted
Beginner
1 View

Hi Kevin

Jump to solution

Hi Kevin

Thanks for your help so far.

I managed to make an example which behaves like my code. The problem seems to occur due to the first record written to the files. I.e. I observe the following with the example below.

1. Decreasing the real array size from 100000 to 10000 removes the problem
2. Using the ivf compiler in version 16 removes the problem
3. Including recl with a sufficient size in the open statement removes the problem (unit 2)
4. Leaving out the first record (the 4 character string) removes the problem

If something is wrong with the code, please let me know.

program recl_test
      
  implicit none
      
  ! Variables
  integer :: i
  real(8) :: a(100000)
  character(LEN=4) :: c
      
  a(:) = 2.0
      
  open(unit=1,file='C:\TEMP\test1.dat', action='readwrite', &
      access='sequential',form='unformatted',status='unknown')
  
  open(unit=2,file='C:\TEMP\test2.dat', action='readwrite', &
      access='sequential',form='unformatted',status='unknown', recl=200000)
      
  !WRITE
  write(1) 'ABCD'
  write(1) a
  
  write(2) 'ABCD'
  write(2) a
      
  !READ
  rewind 1
  rewind 2

  read(2) c
  read(2) a
  
  read(1) c
  read(1) a
      
  close(1)
  close(2)
    
end program recl_test

 

0 Kudos
Highlighted
Beginner
1 View

Maybe I should add that

Jump to solution

Maybe I should add that writing to the file works well. I have compared two files from the version 16 and 17 compiler respectively. These files are identical. I.e. it is the reading of these files that behaves differently.

0 Kudos
Highlighted
Employee
1 View

Thank you Thomas for the

Jump to solution

Thank you Thomas for the reproducer. We are still missing/overlooking something here. The testcase behaves for me when built for both IA-32 and Intel64 w/17.0.1.

What compiler options did you use?   What version of Visual Studio are you using?

0 Kudos
Highlighted
Black Belt
1 View

I see no run time errors with

Jump to solution

I see no run time errors with the code of #7, using the 2017U1 compiler (32-bit or 64-bit) on Windows 10.

One point to note: the unit for the value specified after RECL= is a "file-storage-unit". By default, in Intel Fortran that is four bytes. An -assume option is available to specify the record length in byte units, instead. Other compilers may implement similar variations.

0 Kudos
Highlighted
1 View

Try using the 100000 (failing

Jump to solution

Try using the 100000 (failing array size), remove the recl..., and adding RECORDTYPE='VARIABLE' to your open statements.

I suspect that without the large recl=... that segmented file format was selected .AND. an error developed in writing/reading a multi-segmented record in your test case (this would be a bug). Using RECORDTYPE='VARIABLE' may resolve this for you.

Jim Dempsey

0 Kudos
Highlighted
Beginner
1 View

I have taken a closer look at

Jump to solution

I have taken a closer look at my project settings. The setting causing the problem is the buffered I/O.

/assume:buffered_io

I am using the Microsoft Visual studio 2013 shell version 12.0.40629.00 update 5

0 Kudos
Highlighted
Beginner
1 View

Have you been able to

Jump to solution

Have you been able to reproduce the error yet?

0 Kudos
Highlighted
Employee
2 Views

Sorry for the delay. We still

Jump to solution

Sorry for the delay. We still cannot reproduce it even with that option and your mentioning of it had us wondering if it related to some other reported issues, one in particular is here which I confirmed is fixed in PSXE 2017 Update 2. Without being able reproduce the failure you experienced we could not confirm whether your case was a variant of this or another known case.

If you have your original test case in a project, can you share the build log for that for the case that fails?  Perhaps there is other compiler options being overlooked.

I don’t know whether you want to considering trying PSXE 2017 Update 2 on your end to see whether that may fix the issue or not.

View solution in original post

0 Kudos
Highlighted
Beginner
1 View

Hi Kevin

Jump to solution

Hi Kevin

I now got the time for updating to PSXE 2017 update 2. That solved the problem.

Thanks for your help everybody.

0 Kudos
Highlighted
Employee
1 View

I'm glad to hear that.

Jump to solution

I'm glad to hear that. Perhaps it is a variant of another reported defect then. Please keep us posted should you run into any other related issues moving forward with Update 2.

0 Kudos