Corrupted unformatted file when adding a record

[Petr_Parik]

I have discovered a strange behavior of my legacy application when certain input data size reaches a certain limit. I have tracked the problem to a WRITE statement which causes the corruption of the previous record in an unformatted file.

Illustration F77 code:

      REWIND IDISC
      DO I=1,NREC
        READ(IDISC)
      END DO
      WRITE(IDISC) (IA(I),I=1,LA)

What happens is that the file IDISC is rewinded and NREC records are skipped, then a new record is written. This works normally as expected unless the last (NREC-th) record exceeds 131,072 bytes. If the last record is longer, the READ(IDISC) statement does not skip the whole record (LA*4+8 bytes) but only something like (LA*4+8 mod 131072) bytes, resulting in the new record being written in the middle of the last record causing a corruption.

After some playing I have discovered that if I read the whole last record (not skip it), i.e., something like READ(IDISC) IDATA(LDATA) , the new record is written correctly and no corruption happens. Of course I need to know what is written in the last record which was not hard in this specific case, but I wanted a general fix.

Digging deeper I discovered this can be solved by setting the env. variable FORT_BLOCKSIZE=1048576 (i.e., some value big enough to handle the "long" last record).

I think this might be a bug in the Intel Fortran RTL implementation of the READ statement. However, the bug manifests itself only when using certain optimizations, in my case the -fast option.

[Ron_Green]

What is the version of your compiler? and which OS? 

[Petr_Parik]

Confirmed on:

• ifort 2021.10.0 / Windows 10
• ifort 2021.12.0 / Fedora 35
• ifort 2021.11.0 / Fedora 40

The corruption occurs with the -fast, -O3, -O2 or -O1 options. The corruption does not occur with the -O0 or -debug options.

[Petr_Parik]

Here is the working example: https://www.mirrored.to/files/YLUJXRYL/blocksize.zip_links .

If you make and run the program rpd3 you get

forrtl: severe (24): end-of-file during read, unit 8, file /....../blocksize.P

If you compile without optimizations, it works without any errors. Or, if you add BLOCKSIZE=1048576 to the OPEN statement on line 110.

[TobiasK]

Hi @Petr_Parik

I am not able to construct a working reproducer from your description. Also we are not allowed to access those links. Can you either put your reproducer into a more company friendly environment, e.g. github or create a small reproducer that you can just write down here?

best

Tobias

[Petr_Parik]

Hi Tobias,

It is a part of a proprietary code so putting it on Github is out of the question. But it seems I can attach the reproducer here.

There are two OPT= lines in the makefile; the first (with optimizations) exhibits the error, the second (without optimizations) runs okay.

Please let me know if you need anything else.

Best regards,

Petr

[TobiasK]

@Petr_Parik

this is quite huge, can you please try to reduce the size of the reproducer? It will take a considerable time to go through that reproducer.

[Petr_Parik]

@TobiasK I have removed more unnecessary code. Please find attached a CBFortran project.

The problem occurs when writing to the file IDP in subroutines AIEA1 and/or AINA1, called from subroutine TASV.

[TobiasK]

@Petr_Parik

sorry but your reproducer is still much too complex. Please try to reduce it, then we can have a second look at it.

[Petr_Parik]

@TobiasK The only approach I can think of is to make a memory dump (the program state is stored in 1 large and 1 small array, so it should work), then make a reduced version which reads this dump and then calls the offending subroutine. But it will take me some time to prepare it.