Stream access hangs reading 8-bit integers

[Dave_Allured]

This program should open a file, read two chunks with stream access, and print three messages.  With recent Intel compilers, it hangs on the second read until terminated.

   integer(kind=selected_int_kind(2)) buf(17000)
   integer ios
   open (10, file='a.txt', access='stream', status='old', iostat=ios)
   print *, 'Open file,   ios = ', ios
   read (10, pos=1, iostat=ios) buf
   print *, 'First read,  ios = ', ios
   read (10, pos=17000, iostat=ios) buf(17000)
   print *, 'Second read, ios = ', ios
   end

Results:

[hfe12]$ ifort --version | grep ifort
ifort (IFORT) 19.1.2.254 20200623

[hfe12]$ ifort -g -O0 -traceback -check all -warn all stream.f90

[hfe12]$ ./a.out 
 Open file,   ios =            0
 First read,  ios =            0
^Cforrtl: error (69): process interrupted (SIGINT)
Image              PC                Routine            Line        Source             
a.out              0000000000404BBB  Unknown               Unknown  Unknown
libpthread-2.17.s  00007F5275CAD630  Unknown               Unknown  Unknown
a.out              0000000000440C1C  Unknown               Unknown  Unknown
a.out              000000000041286C  Unknown               Unknown  Unknown
a.out              0000000000411020  Unknown               Unknown  Unknown
a.out              0000000000403C32  MAIN__                      7  stream.f90
a.out              00000000004038A2  Unknown               Unknown  Unknown
libc-2.17.so       00007F52756EE555  __libc_start_main     Unknown  Unknown
a.out              00000000004037A9  Unknown               Unknown  Unknown
[hfe12]$ 

Input is an ordinary ascii file, 17700 bytes, attached.  Contents are not relevant because this is supposed to be binary stream I/O.  The intended data type in line 1 is 8-bit signed integers.  Position 17700 is read twice, and this is intentional.

The chunk size is significant.  I did not narrow it down exactly.  But the problem persists with larger chunk size (and larger files), and it seems to go away when the chunk size and the second position specifier are both less than about 16 K.

I get the same behavior on several platforms and recent compiler versions: Mac OS, Linux; ifort versions 18.0, 19.0, 19.1.  The filesystems are diverse, at least one of them presenting as NFS.  GCC compiles and runs this program as expected, with three output messages and no hangs.

* Is this a legal fortran program?

* Is this a bug in the Intel compiler and runtime?

[Dave_Allured]

Correction.  The position number that is read twice is 17000, not 17700, obviously.

[Arjen_Markus]

I can confirm this phenomenon - I used Intel Fortran 2018. And in this case the program also fails with default integers. Running the same program built with gfortran works as intended.

[mecej4]

The bug can be reproduced with Intel Fortran 2013SP1 on Windows, as well.

Here is another property that may provide some clues. Take the modified test program

   program xdal
   integer, parameter :: FSIZ = 16384
   integer(kind=selected_int_kind(2)) buf(FSIZ)
   integer ios
! The file a.txt can contain anything, but make it longer than FSIZ bytes to see the bug   
   open (10, file='a.txt', access='stream', status='old', iostat=ios)
   print *, 'Open file,   ios = ', ios
   read (10, pos=1, iostat=ios) buf
   print *, 'First read,  ios = ', ios
   read (10, pos=FSIZ, iostat=ios) buf(FSIZ)
   print *, 'Second read, ios = ', ios
   end

The file a.txt should be longer than FSIZ. You could even copy an EXE file to "a.txt" as long as it is longer than FSIZ.

When FSIZ is set to 16384 as shown in the listing, the hang-up occurs. When FSIZ = 16383 = Z'00003FFF', the compiled program runs to completion normally.

[jimdempseyatthecove]

Dave,

Using the failing program that you showed in post #1 (iow build, run, make sure it fails)

Then modify the 1st statement such that it reads ...buff(17000+1)

but leave the other 17000's as-is.

If that succeeds, then you may have an easy work around (declare buffer a bit larger than used).

Jim Dempsey

[Dave_Allured]

Jim, thank you for the suggestion.  Results follow.  However, my sole reason for this bug report is to improve the quality of the Intel compiler product.  My current workaround is to use GCC.  This is not optimal because Intel is the preferred compiler on the supercomputers that I use.

Did you mean this?  The program fails exactly the same way.

   integer(kind=selected_int_kind(2)) buf(17000+1)

Here is more evidence that this is an I/O driver problem, not local memory management.  I switch to an independent scalar variable in lines 1 and 7.  This version also hangs.

   integer(kind=selected_int_kind(2)) buf(17000), b2
   integer ios
   open (10, file='a.txt', access='stream', status='old', iostat=ios)
   print *, 'Open file,   ios = ', ios
   read (10, pos=1, iostat=ios) buf
   print *, 'First read,  ios = ', ios
   read (10, pos=17000, iostat=ios) b2
   print *, 'Second read, ios = ', ios
   end

[Dave_Allured]

Thanks to Arjen and mecej4 for expanded testing.  Interesting results.  I am not surprised that this problem is more widespread.

This is a minimal reproducer, based on one of my old application programs.  I find a note that this method of reading a final byte was working correctly 11 years ago, with Intel 11.1 update 4.  I think a few more recent Intel versions were also working, but confirming that would be time consuming.  The main point here, FWIW, is that this looks like a regression.

According to my notes, it was necessary to specify -assume byterecl to compile this with Intel 11.  So if anyone wants to test with older Intel version, use -assume byterecl.  That option does not seem to be needed with more recent versions.  I tried it both ways, and there is no difference.

In any case, these simple read statements should never hang.  They should always return immediately with a valid status code.

[Barbara_P_Intel]

I'm willing to take a look at this and file a bug report.  Or did you file an issue at the Online Service Center already? It's not my turn to watch those issues, so I'm not sure.

If I am pursuing this, read on...

Can you please attach the input file, a.txt? The initial message implies one was attached, but I don't see it. I just want to be sure I am duplicating your problem.

What compiler options do you use? 

Have you tried the current compiler release 2021.1.2?  It's part of the oneAPI Toolkits. Install oneAPI Base Toolkit and oneAPI HPC Toolkit to have the same features as Parallel Studio XE.

Thank you.

[Dave_Allured]

Barbara, thank you for the help.  I thought I had attached the input file, but I do not find it here.  I will attach it to this message.

I do not have paid access to the service center, therefore I was not able to file an issue there.  This is my compile command line:

ifort -g -O0 -traceback -check all -warn all stream.f90

I will also attach a copy of the last posted version of my demo program, as posted above.  This version avoids confusion about the re-use of an array in two different read statements.

I was not able to try the current compiler release 2021.1.2.  I work on corporate systems, and it is tedious to get updates.  The last version that I can access is 19.1.2.254 20200623.  If you could try the current version with my demo, and report the result here, that would be great.

[Barbara_P_Intel]

Thank you!   I duplicated what you see with the current compiler 2021.1.2.

I did some playing around with the size of the buf array.  16384 is the sweet spot, the amazing power of 2.  If the size of buf is >=16384, the second read hangs.  Less than that and it runs to a normal completion.

I filed a bug on your behalf, CMPLRLIBS-33278.

[Barbara_P_Intel · Accepted solution]

This issue with stream I/O hanging is fixed in the latest compiler, 2021.3.0. I just tested it.  2021.3.0 was released last week.  

Please try it and let me know how it works for you!

[Dave_Allured]

Barbara, I can confirm that ifort version 2021.3.0 runs this test program correctly, and the bug is fixed.  Thanks again for filing this report with the service center.  Sorry for the long delay, it was a complicated year.

---

@Barbara_P_Intel wrote on 2021 July 7:

This issue with stream I/O hanging is fixed in the latest compiler, 2021.3.0. I just tested it.  2021.3.0 was released last week.  

Please try it and let me know how it works for you!

---