Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
28847 Discussions

Corrupted unformatted file when adding a record

Petr_Parik
Beginner
1,005 Views

I have discovered a strange behavior of my legacy application when certain input data size reaches a certain limit. I have tracked the problem to a WRITE statement which causes the corruption of the previous record in an unformatted file.

Illustration F77 code:

 

      REWIND IDISC
      DO I=1,NREC
        READ(IDISC)
      END DO
      WRITE(IDISC) (IA(I),I=1,LA)

 

What happens is that the file IDISC is rewinded and NREC records are skipped, then a new record is written. This works normally as expected unless the last (NREC-th) record exceeds 131,072 bytes. If the last record is longer, the READ(IDISC) statement does not skip the whole record (LA*4+8 bytes) but only something like (LA*4+8 mod 131072) bytes, resulting in the new record being written in the middle of the last record causing a corruption.

After some playing I have discovered that if I read the whole last record (not skip it), i.e., something like READ(IDISC) IDATA(LDATA) , the new record is written correctly and no corruption happens. Of course I need to know what is written in the last record which was not hard in this specific case, but I wanted a general fix.

Digging deeper I discovered this can be solved by setting the env. variable FORT_BLOCKSIZE=1048576 (i.e., some value big enough to handle the "long" last record).

I think this might be a bug in the Intel Fortran RTL implementation of the READ statement. However, the bug manifests itself only when using certain optimizations, in my case the -fast option.

0 Kudos
9 Replies
Ron_Green
Moderator
952 Views

What is the version of your compiler? and which OS? 

0 Kudos
Petr_Parik
Beginner
917 Views

Confirmed on:

  • ifort 2021.10.0 / Windows 10
  • ifort 2021.12.0 / Fedora 35
  • ifort 2021.11.0 / Fedora 40

The corruption occurs with the -fast, -O3, -O2 or -O1 options. The corruption does not occur with the -O0 or -debug options.

0 Kudos
Petr_Parik
Beginner
726 Views

Here is the working example: https://www.mirrored.to/files/YLUJXRYL/blocksize.zip_links .

If you make and run the program rpd3 you get

 

forrtl: severe (24): end-of-file during read, unit 8, file /....../blocksize.P

 

If you compile without optimizations, it works without any errors. Or, if you add BLOCKSIZE=1048576 to the OPEN statement on line 110.

 

 

0 Kudos
TobiasK
Moderator
583 Views

Hi @Petr_Parik

I am not able to construct a working reproducer from your description. Also we are not allowed to access those links. Can you either put your reproducer into a more company friendly environment, e.g. github or create a small reproducer that you can just write down here?


best

Tobias


0 Kudos
Petr_Parik
Beginner
520 Views

Hi Tobias,

It is a part of a proprietary code so putting it on Github is out of the question. But it seems I can attach the reproducer here.

There are two OPT= lines in the makefile; the first (with optimizations) exhibits the error, the second (without optimizations) runs okay.

Please let me know if you need anything else.

Best regards,

Petr

0 Kudos
TobiasK
Moderator
446 Views

@Petr_Parik


this is quite huge, can you please try to reduce the size of the reproducer? It will take a considerable time to go through that reproducer.


0 Kudos
Petr_Parik
Beginner
388 Views

@TobiasK I have removed more unnecessary code. Please find attached a CBFortran project.

The problem occurs when writing to the file IDP in subroutines AIEA1 and/or AINA1, called from subroutine TASV.

0 Kudos
TobiasK
Moderator
205 Views

@Petr_Parik


sorry but your reproducer is still much too complex. Please try to reduce it, then we can have a second look at it.


0 Kudos
Petr_Parik
Beginner
75 Views

@TobiasK The only approach I can think of is to make a memory dump (the program state is stored in 1 large and 1 small array, so it should work), then make a reduced version which reads this dump and then calls the offending subroutine. But it will take me some time to prepare it.

0 Kudos
Reply