BACKSPACE issue with latest Intel Fortran compiler

[Prasha]

Hello,

I am facing an issue with latest Intel 2019 fortran compiler.

I am attaching the fortran code, which reproduces the issue.

In our numerical application, sometimes the user wants to recompute the calculation from a particular step.

Here, we are essentially rewriting records, from that particular step. This is done by reading the output (result) file, till that step number is reached and then fortran BACKSPACE statement  is called, to position the file at the correct record, from which they are overwritten.

In the new Intel 2019, calling WRITE statement, after the BACKSPACE statement, crashes the application, with the following error:

forrtl: An attempt was made to move the file pointer before the beginning of the file.

forrtl: severe (38): error during write, unit 30, file D:\temp\fortran_file_reading\test.dat

The same code however works file in intel 16.0  and before. I will really appreciate any help in understanding this issue

Thanks

Prashanth

[mecej4]

You have two OPEN statements with ERR=nn clauses, but the nn is a label in an IF construct. Branching to such a label from outside the construct is not permitted by the Fortran standard, but appears to be allowed by Intel Fortran.

I could not reproduce the error that you reported using IFort 19.1.3.311  on Windows on the code that you attached. Did you encounter the error in your full application, or with the test code that you provided? Are you sure that at the time of updating the file will always contain a matching record triplet, and that the triplets occur with increasing values of the I value?

[Prasha]

I have modified the code and put ERR=nn in same block as OPEN statement. 

With the previous code attached, we can reproduce the crash in release mode.

To reproduce it in debug mode, I have modified the WRITE statement as

WRITE(IUNIT) (VALUES(IT),IT=1,ISIZE)

to

WRITE(IUNIT) VALUES

and attaching it.

Thanks for your time

Regards

Prashanth

[mecej4]

Your original as well as the revised program run with no errors with the following versions:

19.0.5.281 Build 20190815 (targeting 64 bit)

19.1.3.311 Build 20201010_000000 (targeting 32 and 64-bit)

IFORT 2021.2.0 Build 20210228_000000 (32 and 64-bit)
IFX 2021.2.0 Beta Build 20210317 (64-bit)

The output file is about 205 MB long, so if there is not enough space on the drive on which the current directory is located, there could be problems.

Please report the exact version of the compiler that you are using, and all the options used when compiling and linking. "latest Intel Fortran compiler" is not specific enough. Latest from whose point of view?

If you are using Visual Studio to build, provide a copy of the build log file.

[Prasha]

hello,

Sorry for not being clear about the compiler version. It is Intel(R) Visual Fortran Compiler 19.1.3.311 [Intel(R) 64]

I am attaching the build log, both debug and release.

There is enough space in the disc, about 600gb free

[JohnNichols]

Intel® Fortran Compiler – toolkit version: 2021.2.0, extension version 19.2.0061.16, Package ID: w_oneAPI_2021.2.0.243,

It runs in all versions of 32 and 64 bit for this compiler in about 3 seconds using stock standard VS 2019 preview. 

[JohnNichols]

Just out of interest you are not using LINUX are you?

[JohnNichols]

it runs and compiles on this old Fortran compiler using ifort with no switches and from the rar are you using Windows - sorry for the stupid question. 

It generates the same file each time.

[Prasha]

I am using windows with Visual Studio Professional 2019 version 16.7.10

I am attaching the image of the call stack trace and the dll's libifcoremd.dll and libifconremdd.dll in which this crash is happens.

[JohnNichols]

The attached solution folder works.  Could you see if it runs on your machine -- just a suggestion 

You could also update to the latest VS Preview and the oneAPI and give it a try 

[JohnNichols]

Zip and ship your whole solution folder - another suggestion 

[mecej4]

I can now reproduce the bug. It is caused by using the /assume:buffered_io option, and when ISIZE in the program is set to 32768 or larger.

The bug is also present in the latest (2021.2.0) IFort and IFx OneAPI compilers.

[JohnNichols]

@mecej4  -- why would you use buffered io?   - I understand the theory, but it causes some interesting issues at times, you see problems with it in C# when you need to tell the buffer to flush?

[Prasha]

The ISIZE in our application are usually in millions ( ~30 million ) and might increase in near future. So what is the next step? Will this bug be fixed in the compiler?

Also, does this crash happen only if I use BACKSPACE or can it happen anytime?

Regards

Prashanth

[mecej4 · Accepted solution]

I am confident that the Intel compiler team will fix this bug, since it occurs in the latest OneAPI compilers, as well.

If you have a current license, you can file a bug report at the Intel Service Center . If not, we wait until an Intel representative reads this thread and files a bug report on your behalf.

I am not able to answer your questions, since the answers will depend on the algorithms and conventions used in the compiler and its RTL. You may, if you wish, experiment with some of the options such as /assume:buffered_io , and see if there are facilities (probably non-standard) in the OPEN statement to use a more fine-grain approach to buffering. That is, apply it only a portion of your program other than the one in which you do read-backspace-write operations. Or, if you wish to have your program work as is, you can accept the performance degradation that not specifying buffered I/O may entail.

[Prasha]

Ok, thanks a lot for your time and patience. Since we have the license, i will try to file a bug.

[JohnNichols]

The Intel stuff on io buffers outlines the strategy between on and off, the difference is space and time, but if you read the comments, it would appear that they suggest off as the better solution. 

Interesting problem.  And you learn something every day. 

[Steve_Lionel]

Buffered I/O, which is handled by the run-time library and not the compiler, is very tricky to get right, as is BACKSPACE. (Many years ago I implemented an optimized way to do BACKSPACE for VAX FORTRAN, but I didn't have to deal with buffered I/O at the time.) 

I/O buffering came about as programs and datasets got larger, and significant performance could be gained by collecting data from multiple operations and transferring it to/from disk in a single operation. But some programs perform worse with buffering due to their execution pattern. The Intel library tries to thread the needle, guessing which strategy works best. There have been bugs in the implementation more than once over the years. BACKSPACE makes the problem much harder, and it does not at all astonish me that there are edge cases that fail.

[Prasha]

@Steve_Lionel Say I will not use BACKSPACE, is there a chance that WRITE or READ statements might crash, with /assume:buffered_io option ON?

Also, is there anywhere performance comparison between /assume:buffered_io and nobufffered_io?

( I can anyway check this myself )

Regards

Prashanth

[JohnNichols]

https://software.intel.com/content/www/us/en/develop/documentation/fortran-compiler-oneapi-dev-guide-and-reference/top/language-reference/data-transfer-i-o-statements/forms-for-read-statements/forms-for-sequential-read-statements/rules-for-unformatted-sequential-read-statements.html 

The method is explained in the link.  

no buffering will optimize for time under one rule and for size otherwise. 

I would just run with the no and look at the time. 

[Steve_Lionel]

@Prasha , as John says, you need to test for yourself. It helps some patterns, hurts others. As for risk of crash without BACKSPACE, I'd say that's unlikely now, but it was a serious issue a few years ago when "heuristic buffering" was added.

BACKSPACE is very difficult to do quickly in a variable-length formatted file. It's not too bad on an unformatted file. If you're using BACKSPACE to reprocess a record, consider alternatives such as saving the record contents in a variable.