Unformatted IO and buffering

[Roman1]

I am using Fortran Compiler 17.0.2.187.

I think I have come across a bug in the compiler.  The following test program writes out data to an unformatted file.  It then reads it back in, and compares with the original data to make sure the IO was done correctly.  If I use the default options, the test passes.  However, if I enable buffering with /assume:buffered_io , the test fails.

Roman

   
module unformattedIO_mod

   integer,parameter,private:: mcell=150000

   integer(kind=8),allocatable,private:: wtz(:),  gdiag(:)
   integer(kind=8),allocatable,private:: wtz2(:), gdiag2(:)

   character(len=*),parameter:: testfilename='testIO.dat'

   contains

   !---------------------------------------------------------------------------------

   subroutine writefile()
   implicit none
   integer i
   
      allocate(wtz(mcell), gdiag(mcell) )
      do i = 1, mcell
         wtz(i) = i
         gdiag(i) = 100 * i
      end do

      open(unit=8, file=testfilename, form='unformatted', action='write')
   
      write(8) wtz
      write(8) gdiag

      close(8)
      
   return
   end subroutine writefile

   !----------------------------------------------------------------

   subroutine readfile()
   implicit none

      allocate(wtz2(mcell), gdiag2(mcell) )

      open(unit=9, file=testfilename, form='unformatted', action='read')

      read(9) wtz2
      read(9) gdiag2
   
      close(9)
      
      if ( all(gdiag == gdiag2) .and. all(wtz == wtz2) ) then
         write(*,*) 'Test passed.'
      else
         write(*,*) 'Test failed.'
      end if
   
   return
   end subroutine readfile

end module unformattedIO_mod

!=============================================================

program testUnformattedIO

use unformattedIO_mod, only: writefile, readfile
implicit none

call writefile()
call readfile()

stop
end program testUnformattedIO

[Kevin_D_Intel]

This appears to be a regression with the 17.0.2 compiler. The failure does not occur with the 17.0.1 (or earlier) compiler. If you have 17.0.1 available you could use that as a work around, or as you noted do not use buffered I/O, or with buffered I/O the failure is avoidable (at least for the test case provided) by setting the blocksize to 1200000 via the environment variable FORT_BLOCKSIZE.

Thank you for the convenient reproducer. I escalated this to Development.

(Internal tracking id: DPD200419523)

[Justin_v_]

Replicating this comment to ensure you receive it Roman - This is possibly the same as my problem, as I use the buffered I/O option by default.  But, the problem I identify also arises when buffered I/O is suppressed - as indicated by the VERSION2.zip that is included in my post

[Devorah_H_Intel]

The original issue reported in this thread is fixed in the upcoming 18.0 compiler.

Thank you,

[James_Pearson]

I think I just spent a week or so rediscovering this bug in 2017 FORTRAN Update 4.

Devorah - can you verify the below example runs to completion using the new 18.0 compiler when compiled with the buffered IO optimization?

When I test with 2017 FORTRAN Update 4 and the buffered IO optimization it crashes while reading in the file it writes.

      program main
          IMPLICIT NONE
          
      integer :: scencqs = 53
      
      integer, pointer :: myreg(:), myk(:), myi(:), myj(:)
      real, pointer :: cq(:,:)
      real, pointer :: cvol(:), crhot(:)
      real, pointer :: cdt(:), ped(:), ued(:), ged(:), bed(:), sob(:)
      real, pointer :: pgd(:), pxd(:), ppd(:), pud(:)
      integer*4 :: lun,ic
      integer :: nctot = 0
      
      integer, pointer :: wmyreg(:), wmyk(:), wmyi(:), wmyj(:)
      real, pointer :: wcq(:,:)
      real, pointer :: wcvol(:), wcrhot(:)
      real, pointer :: wcdt(:), wped(:), wued(:),
     &                 wged(:), wbed(:), wsob(:)
      real, pointer :: wpgd(:), wpxd(:), wppd(:), wpud(:)
      integer*4 :: luno,wic
      integer :: wnctot
      
      print *, "Writing file..."
      
      luno=10
      open(unit=luno,form='unformatted',
     &file='test.input')
      
      wnctot=71445
      
      allocate(wmyreg(wnctot),wmyk(wnctot),wmyi(wnctot),wmyj(wnctot),
     &      wcq(scencqs,wnctot),wcvol(wnctot),wcrhot(wnctot),
     &      wcdt(wnctot),wged(wnctot),wbed(wnctot),
     &      wsob(wnctot),wped(wnctot),wued(wnctot))
      allocate(wpgd(wnctot),wpxd(wnctot),
     &      wppd(wnctot),wpud(wnctot))
      
      write(luno) (wmyreg(wic),wmyk(wic),wmyi(wic),wmyj(wic),
     &      wic=1,wnctot)
      write(luno) (wcq(1:scencqs,wic),wic=1,wnctot)
      write(luno) (wcvol(wic),wcrhot(wic),wcdt(wic),wged(wic),
     &      wbed(wic),wsob(wic),wped(wic),wued(wic),wic=1,wnctot)
      write(luno) (wpgd(wic),wpxd(wic),wppd(wic),wpud(wic),
     &      wic=1,wnctot)
      call flush(luno)
      close(luno)
          
      print *, "Reading file..."
            
      lun = 10
      open(unit=luno,form='unformatted',
     &file='test.input')
      
      nctot=71445
      
      allocate(myreg(nctot),myk(nctot),myi(nctot),myj(nctot),
     &      cq(scencqs,nctot),cvol(nctot),crhot(nctot),
     &      cdt(nctot),ged(nctot),bed(nctot),
     &      sob(nctot),ped(nctot),ued(nctot))
      allocate(pgd(nctot),pxd(nctot),
     &      ppd(nctot),pud(nctot))
      read(lun) (myreg(ic),myk(ic),myi(ic),myj(ic),ic=1,nctot)
      read(lun) (cq(1:scencqs,ic),ic=1,nctot)
      read(lun) (cvol(ic),crhot(ic),cdt(ic),ged(ic),
     &      bed(ic),sob(ic),ped(ic),ued(ic),ic=1,nctot)
      read(lun) (pgd(ic),pxd(ic),ppd(ic),
     &      pud(ic),ic=1,nctot)

      write(*,*) "Finished"
      close(lun)
      
      end program main

[mecej4]

The code in #5 does not follow either the fixed form or the free form of Fortran source. The compiler refuses to accept this nonstandard source form.

[Devorah_H_Intel]

Hi James,

Yes it builds and runs in 18.0

$ ifort -v
ifort version 18.0.0

$ ifort -g -traceback -assume buffered_io test2.f90

$ ./a.out

 Writing file...
 Reading file...
 Finished

[Devorah_H_Intel]

To add: the test fails at runtime on 17.0.4

[Craig_T_]

My organization appears to have been burned by this same problem using 17.0.6.270.

Devorah H., can you elucidate the scope of the problem fixed? Are there any situations where it's known that buffered and unformatted I/O using ifort2017 should still be safe?

Note my organization can not adopt ifort2018 because its run-time performance degraded for our code compared to ifort2017 for still unknown reasons.