Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
28384 Discussions

Regression with "backspace" in 2023.0

krefson
Beginner
2,635 Views

The following program demonstrates an I/O regression with the backspace statement in 2023.0.

program test_backspace
  integer :: test_unit=4
  character(len=16) :: filename = 'test_file'
  integer :: status
  integer :: one = 1, two = 2, three = 3

  open(unit=test_unit, file=filename, form='unformatted',action='write')
  write(test_unit) one, two
  close(test_unit)
  open(unit=test_unit, file=filename, form='unformatted',action='read')
  read(test_unit, iostat=status) one, two, three
  if( status == 0 ) then
     write(*,*) ' Read one more item than written????'
  else
     backspace(test_unit)
     read(test_unit, iostat=status) one, two
     if( status == 0 ) then
        write(*,*) ' Successfully read two items written:',one,two
     else
        write(*,*) ' Failed to read after backspace, code=',status
     end if
  end if
  
end program test_backspace

$module load compiler compiler/2022.2.1

ifort -o test.ifort-2021.7 test-backspace.f90
$ ./test.ifort-2021.7
 Successfully read two items written:           1           2
$ module switch compiler compiler/2023.0.0

$ ifort --version
ifort (IFORT) 2021.8.0 20221119
Copyright (C) 1985-2022 Intel Corporation.  All rights reserved.
$ ifort -o test.ifort-2021.8 test-backspace.f90
$ ./test.ifort-2021.8
 Failed to read after backspace, code=          -1

ifx behaves in an identical manner.


KR.


 

0 Kudos
1 Solution
Steve_Lionel
Honored Contributor III
2,296 Views

I've been involved in a discussion of this before. EOR applies only for a non-advancing READ. In an advancing READ, which this, is, trying to read past the end of the record is an error, not an EOR condition.  The standard says, "If an error condition occurs during execution of an input/output statement, the position of the file becomes indeterminate." Therefore, you can't reliably BACKSPACE to recover from this. Note that EOR is not an error.

That INQUIRE changes the behavior is unfortunate, but the example program here is relying on undefined behavior. The only way to recover from this is to REWIND the file and then start reading again.

View solution in original post

26 Replies
Ron_Green
Moderator
2,194 Views

Definitely a regression.  We touched that code back in October to fix some behavior on the console units to conform to the Fortran Standard.  We must've introduced an error.  

And yes, IFX and IFORT share the same Fortran Runtime Library so we expect IO to behave the same in both - in this case "same" being wrong behavior.

 

I've modified the code slightly to help our developers.  And I discovered that if I put in an INQUIRE before the BACKSPACE there is no error! Very very strange.  Compile and run this 2 ways:

 

ifort qnew5.F90
./a.out

ifort -DDEBUG qnew5.F90
./a.out

 

code

 

cat qnew5.F90
program test_backspace
  integer :: test_unit=42
  character(len=16) :: filename = 'test_file'
  character(len=200) :: error_msg = ''
  integer :: status, whatsnext
  integer :: one = 1, two = 2, three = 3

  open(unit=test_unit, file=filename, form='unformatted',action='write')
  write(test_unit) one, two
  close(test_unit)
  open(unit=test_unit, file=filename, form='unformatted',action='read')
  read(test_unit, iostat=status, iomsg=error_msg) one, two, three
  if( status == 0 ) then
     write(*,*) ' Read one more item than written????'
  else
     write(*,*) ' Tried to read too much, 3 itmes. iostat is  ', status
     write(*,*) '    iomsg is ', trim(error_msg)
#ifdef DEBUG
     inquire(test_unit,nextrec=whatsnext)
     write(*,*) '    nextrec is ', whatsnext
#endif

     write(*,*) '    rewinding unit and will try to read 2 items'
     !... here is the backspace which should put as back to the start of the record
     backspace(test_unit)
     read(test_unit, iostat=status, iomsg=error_msg) one, two
     if( status == 0 ) then
        write(*,*) ' Successfully read two items written:',one,two
     else
        write(*,*) ' Failed to read after backspace, code=',status
        write(*,*) '    iomsg =', trim(error_msg)
     end if
  end if
  
end program test_backspace

 

I'll get a bug report open on this.  We're already checking over the code to see what we did wrong. 

 

thanks for finding this. 

0 Kudos
Ron_Green
Moderator
2,105 Views

I forgot to list the bug ID for this one: it's CMPLRLIBS-34238


0 Kudos
Stephen_Sutcliffe
New Contributor II
1,982 Views

Hi Ron,

 

Have you any information regarding the availability of a fix for this?

For now I'll have to go back to previous version as this regression is affecting my application.

I do have premier support subscription so should be able to download the older versions. If I do can I just download the fortran classic only without the oneAPI without affecting integration with VS 2019?

I don't really have enough disk space to keep lots of versions of the full oneAPI stuff. I was hoping to switch to IFX soon but there still seems to be too many teething problems cropping up to do so with confidence.

Thanks,

Steve

0 Kudos
Stephen_Sutcliffe
New Contributor II
1,966 Views

What does this message mean? I get it when trying to download the, what appears to be, the penultimate version of the IFORT fortran compiler.

Stephen_Sutcliffe_0-1673879748822.png

I have subscribed to premier support (or equivalent) since the days of Digital Fortran.

 

I need to download a previous version as the latest (Dec 2023) has a regression which causes BACKSPACE to not function properly in IFORT & IFX as mentioned in previous reply.

My subscription is not due to expire until Nov 2023 so don't see why I need to pay for 'backdated renewal' of retrospective periods that I had already for continuously?

 

Thanks 


Steve

0 Kudos
andrew_4619
Honored Contributor II
1,961 Views

You don't, I used to get some nonsense like this with the licence number sometimes changing on renewal. Raise a ticket on it at licencing.

0 Kudos
JohnNichols
Valued Contributor III
1,948 Views

I am intrigued by a note in your bio about filament winding.  How would you use Fortran in this application?  

Looks challenging?  

John

0 Kudos
andrew_4619
Honored Contributor II
1,940 Views

Try Cadfil.com or FilamentWindingFea.com

0 Kudos
Ron_Green
Moderator
1,899 Views

We had a lot of discussion on this issue in the Fortran Runtime team.

 

The standard says:

2 12.3.4.4 File position after data transfer
3 1 If an error condition (12.11) occurred, the position of the file is indeterminate. If no error condition occurred,
4 but an end-of-file condition (12.11) occurred as a result of reading an endfile record, the file is positioned after
5 the endfile record.

And for BACKSPACE:

If the preceding record is an endfile record, the file is positioned before the endfile record.

In this case, the READ of three values got an error (67, input requires too much data). This is like EOR, but it's treated more like EOF, as the current 'next read point' is after the end of the first (and only record), We are unofficially at EOF, but officially we are at a point after the endfile record. This is because in our implementation, the endfile record is virtual, so we remember where we are officially by setting an internal FRTL flag. This means after the endfile record.

A BACKSPACE at this point is defined to put the 'next read point' at EOF officially, so that's what it does when there's no INQUIRE.

Unfortunately, the INQUIRE clears an FRTL flag, so it now looks like we are at the endfile record.  So a BACKSPACE will go back one record. It does that because it does a generic clean-up of the flags on the unit, and the generic clean-up clears "endfile" flag. This is a bug.

 

One way to show this diagnosis is correct is for the user code to do two BACKSPACE-s in a row. The first moves from after the endfile record to the endfile record; the second then moves from the endfile record to the previous record, which is the first record.

 
0 Kudos
Ron_Green
Moderator
1,893 Views

Steve, I sent you direct email regarding the license trouble.  Let's work through this in email.  

0 Kudos
Ron_Green
Moderator
1,887 Views

Interestingly, I had hoped that ISO_FORTRAN_ENV could clean this up a bit.  But alas, both gfortran and ifort (all versions) in this case return an IOSTAT that is neither IOSTAT_EOR or IOSTAT_END.  

after the 3 item read, both compilers are off the end of the file.  gfortran returns IOSTAT 5016.  IFORT 67.  These are not in ISO_FORTRAN_ENV as constants.  So I made my own parameters for them (see the below).

 

here is the code I used to work around.  Note I moved the BACKSPACE into a conditional around IOSTAT, branching on gfortran vs ifort.  and for ifort, a workaround: do 1 backspace.  inquire to check next record.  if 0 or less backspace again.

 

program test_backspace
  use iso_fortran_env
  implicit none

  integer, parameter :: GFORTRAN_PAST_EOF = 5016
  integer, parameter :: IFORT_PAST_EOF = 67

  integer :: test_unit=42
  character(len=16) :: filename = 'test_file'
  character(len=200) :: error_msg = ''
  integer :: status=0, whatsnext
  integer :: one = 1, two = 2, three = 3

  open(unit=test_unit, file=filename, form='unformatted',action='write')
  write(test_unit) one, two
  close(test_unit)
  open(unit=test_unit, file=filename, form='unformatted',action='read')
  read(test_unit, iostat=status, iomsg=error_msg) one, two, three
  if( status == 0 ) then
     write(*,*) ' Read one more item than written????'
     write(*,*) '    iomsg is ', trim(error_msg)
  else if ( status == GFORTRAN_PAST_EOF ) then
          backspace(test_unit)
  else if ( status == IFORT_PAST_EOF ) then
          backspace(test_unit)
          inquire(test_unit,nextrec=whatsnext)
          if ( whatsnext <= 0 ) then
            !ifort/ifx 2022 and greater needs another backspace
            backspace(test_unit)
          end if     
  else
     write(*,*) ' Tried to read too much, 3 itmes. iostat is  ', status
     write(*,*) '    iomsg is ', trim(error_msg)
  end if
 
  !..we should be positioned back to the previous record
  read(test_unit, iostat=status, iomsg=error_msg) one, two
  if( status == 0 ) then
      write(*,*) ' Successfully read two items written:',one,two
  else
      write(*,*) ' Failed to read after backspace, code=',status
      write(*,*) '    iomsg =', trim(error_msg)
  end if
  
end program test_backspace

 

and the results work "correctly" with gfortran 12, and the older and newer versions of IFORT.  IFX behaves same-as ifort so not shown

 

 

$ gfortran qnew5.F90 
$ ./a.out
  Successfully read two items written:           1           2
$ 
$ rm a.out
$ ifort -V qnew5.F90 
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.8.0 Build 20221119_000000
Copyright (C) 1985-2022 Intel Corporation.  All rights reserved.

 Intel(R) Fortran 2021.8.0-1196
GNU ld version 2.37-37.fc36
$ ./a.out
  Successfully read two items written:           1           2

# and now an old 2021 ifort
$ ifort -V qnew5.F90 
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.1 Build 20201112_000000
Copyright (C) 1985-2020 Intel Corporation.  All rights reserved.

 Intel(R) Fortran 2021.1-2085
GNU ld version 2.37-37.fc36
rwgreen@orcsle153:~/quad/q05695032$ ./a.out
  Successfully read two items written:           1           2

 

 

0 Kudos
Steve_Lionel
Honored Contributor III
2,297 Views

I've been involved in a discussion of this before. EOR applies only for a non-advancing READ. In an advancing READ, which this, is, trying to read past the end of the record is an error, not an EOR condition.  The standard says, "If an error condition occurs during execution of an input/output statement, the position of the file becomes indeterminate." Therefore, you can't reliably BACKSPACE to recover from this. Note that EOR is not an error.

That INQUIRE changes the behavior is unfortunate, but the example program here is relying on undefined behavior. The only way to recover from this is to REWIND the file and then start reading again.

Ron_Green
Moderator
1,868 Views

the INQUIRE behavior was a bug.  That was fixed so that it does affect the state of the unit and it's flags in the RTL. That fix will appear in 2023.1

0 Kudos
Stephen_Sutcliffe
New Contributor II
1,857 Views

Hi Ron,

In my case the BACKSPACE wasn't at the end of a file. The code was supposed the go back and reread a line (single character buffer). But it didn't go back a line and following READ statement read the next line instead.  When I looked at the original post (relating to BACKSPACE Regression) I replaced all BACKSPACE with a subroutine which simply call INQUIRE followed by BACKSPACE so I could try to get a work round which could be called up in place of the standard BACKSPACE.  In this subroutine put the INQUIRE in before BACKSPACE thinking it was a fix whereas this may be a bit of a red herring and possibly be the cause of malfunctioning BACKSPACE.

Am I right in saying that BACKSPACE is actually working correctly in latest version but the INQUIRE is where the regression is?

If that is the case I can simply remove the INQUIRE. 

 

Steve

0 Kudos
Ron_Green
Moderator
1,840 Views

Yes, we had a bug in INQUIRE.  It was changing our FRTL internal state of the unit (file).  I think you will agree that an INQUIRE should NOT change the status or state of a file.

This was fixed, and the fix will show up in 2023.1 and all future compilers.

So yes, I would recommend to not use this bug artifact to get your code to function correctly.

0 Kudos
FortranFan
Honored Contributor II
1,809 Views

@krefson , @Stephen_Sutcliffe :

Just in case you have not considered STREAM IO yet, it may be worth your while to look into whether STREAM access is something you can make use of in your codes.  A couple of users I know who do a lot of IO in their codes (I don't) have come to like STREAM and have told me it is more convenient for their needs.  STREAM may provide you a way to workaround the current regression also, for example:

   character(len=*), parameter :: testfile = "test.txt"
   integer :: x(4)
   call write_file( testfile, nelem=4 )
   x = 0
   call read_file( testfile, x )
   print *, x, "; expected is [1,2,3,4]"
   x = 0
   call read_file( testfile, x(1:2) )
   print *, x(1:2), "; expected is [1,2]"
   call write_file( testfile, nelem=2 )
   x = 0
   call read_file( testfile, x )
   print *, x, "; expected is ??"
contains
   subroutine write_file( file, nelem )
      ! Argument list
      character(len=*), intent(in) :: file
      integer, intent(in) :: nelem
      ! Local variables
      integer, parameter :: dat(*) = [( i, integer :: i = 1,4 )]
      integer :: lun, istat
      open(newunit=lun, file=file, access="stream", form="unformatted", action="write", &
         status="replace", iostat=istat)
      if (istat /= 0) then
         print *, "Error opening file: iostat = ", istat
         stop
      end if
      write( lun ) dat(1:nelem)
      close( lun, iostat=istat )
      if (istat /= 0) then
         print *, "Error closing file: iostat = ", istat
         stop
      end if
   end subroutine 
   subroutine read_file( file, vals )
      use, intrinsic :: iso_fortran_env, only : I8 => int64
      ! Argument list
      character(len=*), intent(in) :: file
      integer, intent(inout) :: vals(:)
      ! Local variables
      integer :: lun, pos, istat, i
      integer(I8) :: sz
      inquire( file=file, size=sz, iostat=istat )
      if (istat /= 0) then
         print *, "Error opening file: iostat = ", istat
         stop
      end if
      open(newunit=lun, file=file, access="stream", form="unformatted", action="read", &
         iostat=istat)
      if (istat /= 0) then
         print *, "Error opening file: iostat = ", istat
         stop
      end if
      loop_read: do i = 1, size(vals) ! read the file stream elementally
         read( unit=lun, iostat=istat ) vals(i)
         if (istat > 0) then
            print *, "Error reading element ", i, " iostat = ", istat
            stop
         end if
         inquire( unit=lun, pos=pos )
         if ( pos >= sz ) then
            ! throw an error
            if ( i < size(vals) ) then 
               print *, "End of stream encountered before all the values are read:" 
               print *, "pos = ", pos, "; file size = ", sz
               error stop
            end if
            exit loop_read
         end if
      end do loop_read
      close( lun, iostat=istat )
      if (istat /= 0) then
         print *, "Error closing file: iostat = ", istat
         stop
      end if
   end subroutine
end 
C:\temp>ifx /standard-semantics p.f90
Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2023.0.0 Build 20221201
Copyright (C) 1985-2022 Intel Corporation. All rights reserved.

Microsoft (R) Incremental Linker Version 14.34.31937.0
Copyright (C) Microsoft Corporation.  All rights reserved.

-out:p.exe
-subsystem:console
p.obj

C:\temp>p.exe
 1 2 3 4 ; expected is [1,2,3,4]
 1 2 ; expected is [1,2]
 End of stream encountered before all the values are read:
 pos =  9 ; file size =  8
0 Kudos
Stephen_Sutcliffe
New Contributor II
1,791 Views

Hi FortranFan,

Thanks for the information. Funnily enough the part in my code, where I thought BACKSPACE wasn't working, used a similar streaming type read function (see below). I believe this was the reason the BACKSPACE was not having any effect. The function is designed to read a character string of variable length and return it's contents and/or length. This function call replaced a previous standard READ statement, with a fixed length character buffer, as the files being read in had increased record lengths in later versions of the application. 

When I saw the post with heading "Regression with BACKSPACE" I assumed, incorrectly, ("never assume") that the actual BACKSPACE function had regressed. The main problem for me was that I couldn't download and install the previous version of compiler to verify if this was the case or not.

function dv_ReadLine(aunit, InLine, trimmed) result(nchars)

!***********************************************************************
! read Line from text file of Unknown Length
! If Buffer not present length of line is returned
!***********************************************************************

implicit none

! Arguments
integer(4), intent(IN) :: aunit
character(LEN=:), allocatable, optional :: InLine
logical(4), intent(in), optional :: trimmed

! Constants
integer, parameter :: line_buf_len = 1024*4

! Local Variables
character(LEN=line_buf_len) :: InS
logical(4) :: OK, set
integer(4) :: nchars
integer(4) :: status, size

OK = .false.
set = .true.
nchars = 0
do
read (aunit,'(a)',advance='NO',iostat=status, size=size) InS
OK = .not. IS_IOSTAT_END(status)
if (.not. OK) then
nchars = -1
return
endif
if (present(InLine)) then
if (set) then
InLine = InS(1:size)
set = .false.
else
InLine = InLine // InS(1:size)
end if
else
nchars = nchars + size
end if
if (IS_IOSTAT_EOR(status)) exit
end do
if (present(trimmed) .and. present(InLine)) then
if (trimmed) InLine = trim(adjustl(InLine))
nchars = len_trim(InLine)
end if

return
end function dv_ReadLine

I think it has been established that the BACKSPACE is working as intended and the actual regression is in the INQUIRE statement.

 

Steve

0 Kudos
krefson
Beginner
1,752 Views

Ron and Steve, Thank you for your responses.

This is a salutory warning about the hazards of relying on nonconforming behaviour even though every one of the many compilers I have tried have supported the obvious extended behaviour until now.

I find it to be an unfortunate lacuna in the standard that there is NO standard-conforming mechanism to read sequential, unformatted records without prior knowledge of the exact length. Frustrating also that possible alternatives like non-advancing read and EOR flags exclude the sequential, unformatted case for no obvious reason. And doubly unfortunate given that the standard implicitly requires the implementation to store the record length information but provides no mechanism to access it!  Time for a submission to J3 perhaps?

K.

 

0 Kudos
Steve_Lionel
Honored Contributor III
1,728 Views

It's generally been the case that in order to read a sequential unformatted file, you need to know how it was written and, ideally, which compiler was used to build the application that wrote it, as there are implementation-dependent physical representations. If you have an application that needs to write a variable number of unformatted fields, you could use stream I/O or write as the first element the number of subsequent elements.

0 Kudos
ATr
Novice
1,703 Views

Dear Sirs,

 

today I installed the Intel® Fortran Compiler Classic 2021.8.0 [Intel(R) 64] and my fortran code that was working since last 10 years

just refused to work; this is related to the iostat in the read statement; here is a MWE

the funny thing is that it does not work in the debug version while in the release it works;

one can bypass the bug by adding   end=.... to the read statement

something like (then iostat -->  -1)

read(191,*,end=100,iostat=err_flag) line

but fundamental question is : is it a bug in the Intel compiler ? if not why this was working since 10 years ?

and last question is how to downgrade to the previous Intel compiler that was free of this bug

 

subroutine bug_IO
character*128 line
integer*4 :: i, err_flag
open (191,file = 'test.txt')
line = 'aaaaaa'
write(191,*) line
close (191)

open(191,file = 'test.txt')
err_flag = 0
do while (err_flag.eq.0)
  read(191,*,iostat=err_flag) line

end do

100 continue
end subroutine bug_IO

 

 

0 Kudos
Stephen_Sutcliffe
New Contributor II
1,683 Views

Did you uninstall the older version before installing the latest? If not you could select the older compiler version in VS Tools->Options->Intel Compilers and Libraries (IFX or IFORT). Again ensure you select appropriate 32 or 64 bit version.

If you have Premier Support you can try sending a request "ticket" for a link to a previous version, a process I went through last week.

Are you using IFX or IFORT?  I haven't experienced any such problems in my programs but I still use IFORT. I don't get my program to successfully build yet using IFX so sticking with the IFORT for now.

You don't say how the error manifests itself. Does it crash or give the wrong iostat code, or not populate the line variable

What happens if you replace the read(191,*,iostat=err_flag) line 

                                                     with  read(191,'(a)',err_flag) line

 

You're best providing the more details to help get to the bottom of your problem.

 

Good luck,

 

Steve

 

 

0 Kudos
Reply