topic One side effect of accessing in Intel® Fortran Compiler

Reading file of length zero

Diehl__Martin — Tue, 02 Jun 2020 07:30:04 GMT

I have a general reading routine that crashes for empty files (ASCII).

The following minimal example shows this behavior (compile; touch empty.txt; ./out) when compiled with ifort version 19.0.2.187. Is that the expected behavior, i.e. would I need to check for fileLength/=0 or is ifort suppost to figure this out.

program test

 character(len=:), allocatable :: a
 a = IO_read('empty.txt')

 print*, len(a)
 print*, '#'//a//'#'

contains

function IO_read(fileName) result(fileContent)

  character(len=*),  intent(in) :: fileName
  character(len=:), allocatable :: fileContent
  integer ::  &
    fileLength, &
    fileUnit, &
    myStat

  inquire(file = fileName, size=fileLength)
  open(newunit=fileUnit, file=fileName, access='stream',&
       status='old', position='rewind', action='read',iostat=myStat)
  if(myStat /= 0) stop 1
  allocate(character(len=fileLength)::fileContent)
  read(fileUnit) fileContent
  close(fileUnit)

end function IO_read

end program test

I tried your program with

Arjen_Markus — Tue, 02 Jun 2020 07:42:22 GMT

I tried your program with gfortran and that worked fine. I am not sure what should be expected acorrding to the standard. The simplest way around this problem is of course to always check for an error. It may be one of those corner cases that are not covered.

The problem is the read, it

andrew_4619 — Tue, 02 Jun 2020 07:43:08 GMT

The problem is the read, it should have an iostat and you should then manage the eof status and other errors, that covers the bases. Opening an empty file is not and error. Reading one is.

Quote:andrew_4619 wrote:

Diehl__Martin — Tue, 02 Jun 2020 07:56:08 GMT

andrew_4619 wrote:
The problem is the read, it should have an iostat and you should then manage the eof status and other errors, that covers the bases. Opening an empty file is not and error. Reading one is.

thanks for the quick reply. Adding iostat indeed works. Its value is -1, but I don't need to `manage` anything.

Might be a corner case, but to me this behavior is unexpected. Why should the use of an optional, intent(out) variable change the behavior of read()?

One more thing I have noticed

Diehl__Martin — Tue, 02 Jun 2020 08:08:35 GMT

One more thing I have noticed:

iostat is 0 if I read a file of length>0 into a string of matching len

iostat is -1 if I read file of length 0 into a string of len 0

gfortran's iostat is 0 in both cases, I find this more consistent

I believe implicit allocation

andrew_4619 — Tue, 02 Jun 2020 08:24:10 GMT

I believe implicit allocation of a string by a read is not standard Fortran.

Quote:andrew_4619 wrote:

Diehl__Martin — Tue, 02 Jun 2020 08:26:46 GMT

andrew_4619 wrote:
I believe implicit allocation of a string by a read is not standard Fortran.

I'm not doing that, fileContent is allocated to size 0:

  allocate(character(len=fileLength)::fileContent)
  read(fileUnit) fileContent

There is some ambiguity here:

Arjen_Markus — Tue, 02 Jun 2020 08:37:24 GMT

There is some ambiguity here:

allocate( character(len=0) :: string )

do 
     read(10, iostat =ierr)  string
     if ( ierr /= 0 ) then
          exit
     endif
enddo

This program would never stop with gfortran, because it never encounters an end-of-file, whereas with Intel Fortran it does.

Of course, this is a trifle contrived, but it does illustrate that there are more sides to the problem!

Conceptually:

andrew_4619 — Tue, 02 Jun 2020 09:54:42 GMT

Conceptually:

1) Is a null file by default at EOF?

2) Is a "null" read a read?

3) does a read check for "eof status" before or after a null read?

I haven't read the standard but I expect some of these are probably undefined and/or OS dependant. A null read is conceptually pointless and I would suggest it is the coders responsibility to avoid/manage such events.

@andrew_4619: Interesting

Diehl__Martin — Tue, 02 Jun 2020 12:14:55 GMT

@andrew_4619: Interesting questions.

I just want to emphasize that I am using the "stream" access mode which might be reason for the strange behavior

1) Is a null file by default at EOF?

When reading a file of length N into a string of length N, EOF is not reached. Therefore, it would be consistent if opening a zero length file would not default to EOF. However, the most consistent situation would be immediate EOF for zero-length files and EOF after reading N characters (or bytes) from an file of length N (in bytes or characters).

2) Is a "null" read a read?

Does not matter, as long as a null read does not change the status

3) does a read check for "eof status" before or after a null read?

The general question would be whether EOF status is reached when the last bit of information is read or when one tries to go beyond this limit.

A null read is conceptually pointless and I would suggest it is the coders responsibility to avoid/manage such events.

The actual read is pointless, but taking care of it is annoying. Reading a file containing N characters results in a string of length N, I dont' see a reason why N > 0 is required since strings of size 0 are totally valid.

Here's my analysis of the

Steve_Lionel — Tue, 02 Jun 2020 14:35:15 GMT

Here's my analysis of the situation...

The READ is attempting to read zero bytes from an empty file connected for unformatted stream access. The standard says that an end-of-file condition occurs when "an attempt is made to read beyond the end of a stream file." Since the READ is not trying to read beyond the end of the file, the file position should remain unchanged and the READ should succeed.

Therefore, I believe this is an error on the part of ifort and should be reported to Intel using the Online Service Center.

"an attempt is made to read

andrew_4619 — Tue, 02 Jun 2020 17:04:11 GMT

"an attempt is made to read beyond the end of a stream file."

Maybe one could also say that an attempt to read nothing is in fact not attempt at all.....

Quote:andrew_4619 wrote:

Steve_Lionel — Tue, 02 Jun 2020 17:41:38 GMT

andrew_4619 wrote:
Maybe one could also say that an attempt to read nothing is in fact not attempt at all.....

I wouldn't say that. There are side-effects of a READ that have nothing to do with the actual reading, such as connecting to a file if not already connected.

One side effect of accessing

cryptogram — Wed, 03 Jun 2020 10:51:35 GMT

One side effect of accessing a zero length file, is that you may run into contention with your anti-malware scanner.

We were using a model that would open an error log file prior to starting a lengthy calculation, and close it at the end of the calculation.

If there were no errors to log, this results in a file with 0 length. The calculations were being performed inside a loop, so the model would

repeatedly open and close this file.

The malware scanner saw these zero length files as suspicious, because a zero length file still reserves a cluster of disk space, which is a good place to

hide malware. Eventually, the model would crash because the malware scanner was scanning the file at just the wrong time.

>>The malware scanner saw

jimdempseyatthecove — Wed, 03 Jun 2020 13:36:17 GMT

>>The malware scanner saw these zero length files as suspicious

That is a crappy way to implement a malware scanner. It is not unusual for a program to pre-open an output file, thus reducing filesystem latency, for use later. Or, open a log file (as you did), only to not log errors. Or, to use the existence or or absence of a file for use in inter-process signaling. All of these uses are completely reasonable. Flagging as potential malware is not correct.

Jim Dempsey

Martin, will you please

Steve_Lionel — Wed, 03 Jun 2020 20:02:01 GMT

Martin, will you please submit a bug report to Intel via https://supporttickets.intel.com/?lang=en-US ? They may not pick up on it from here.

Quote:Steve Lionel (Ret.)

Diehl__Martin — Wed, 03 Jun 2020 20:35:24 GMT

Steve Lionel (Ret.) (Blackbelt) wrote:
Martin, will you please submit a bug report to Intel via https://supporttickets.intel.com/?lang=en-US ? They may not pick up on it from here.

I did that, but the request was closed because I could not provide any license information. I have an educational license and don't know exactly how to figure out who is the officially responsible:

Nikky from Intel Support responded Yesterday 06:29 PM
Martin, our records indicate that you do not have a supported product associated with your account.
You need a supported product in order to qualify for Priority Support. As such, this ticket is being closed.

Sigh. Ok, I will file it for

Steve_Lionel — Thu, 04 Jun 2020 00:51:32 GMT

Sigh. Ok, I will file it for you.

Filed as request number

Steve_Lionel — Thu, 04 Jun 2020 01:00:00 GMT

Filed as request number 04669455. As a workaround, skip the READ if the length is zero.

Quote:Steve Lionel (Ret.)

Diehl__Martin — Thu, 04 Jun 2020 09:03:00 GMT

Steve Lionel (Ret.) (Blackbelt) wrote:
Filed as request number 04669455. As a workaround, skip the READ if the length is zero.

Thanks. Actually, adding iostat already resolves the issue of a segmentation fault. The only small inconsistency is that

iostat /= 0

does not work for zero length files, but

iostat > 0

is also fine for the code I'm using