topic To add: the test fails at in Intel® Fortran Compiler

Unformatted IO and buffering

Roman1 — Thu, 30 Mar 2017 21:34:37 GMT

I am using Fortran Compiler 17.0.2.187.

I think I have come across a bug in the compiler. The following test program writes out data to an unformatted file. It then reads it back in, and compares with the original data to make sure the IO was done correctly. If I use the default options, the test passes. However, if I enable buffering with /assume:buffered_io , the test fails.

Roman

   
module unformattedIO_mod

   integer,parameter,private:: mcell=150000

   integer(kind=8),allocatable,private:: wtz(:),  gdiag(:)
   integer(kind=8),allocatable,private:: wtz2(:), gdiag2(:)

   character(len=*),parameter:: testfilename='testIO.dat'

   contains

   !---------------------------------------------------------------------------------

   subroutine writefile()
   implicit none
   integer i
   
      allocate(wtz(mcell), gdiag(mcell) )
      do i = 1, mcell
         wtz(i) = i
         gdiag(i) = 100 * i
      end do

      open(unit=8, file=testfilename, form='unformatted', action='write')
   
      write(8) wtz
      write(8) gdiag

      close(8)
      
   return
   end subroutine writefile

   !----------------------------------------------------------------

   subroutine readfile()
   implicit none

      allocate(wtz2(mcell), gdiag2(mcell) )

      open(unit=9, file=testfilename, form='unformatted', action='read')

      read(9) wtz2
      read(9) gdiag2
   
      close(9)
      
      if ( all(gdiag == gdiag2) .and. all(wtz == wtz2) ) then
         write(*,*) 'Test passed.'
      else
         write(*,*) 'Test failed.'
      end if
   
   return
   end subroutine readfile

end module unformattedIO_mod

!=============================================================

program testUnformattedIO

use unformattedIO_mod, only: writefile, readfile
implicit none

call writefile()
call readfile()

stop
end program testUnformattedIO

This appears to be a

Kevin_D_Intel — Fri, 31 Mar 2017 10:10:21 GMT

This appears to be a regression with the 17.0.2 compiler. The failure does not occur with the 17.0.1 (or earlier) compiler. If you have 17.0.1 available you could use that as a work around, or as you noted do not use buffered I/O, or with buffered I/O the failure is avoidable (at least for the test case provided) by setting the blocksize to 1200000 via the environment variable FORT_BLOCKSIZE.

Thank you for the convenient reproducer. I escalated this to Development.

(Internal tracking id: DPD200419523)

Replicating this comment to

Justin_v_ — Thu, 06 Jul 2017 20:07:30 GMT

Replicating this comment to ensure you receive it Roman - This is possibly the same as my problem, as I use the buffered I/O option by default. But, the problem I identify also arises when buffered I/O is suppressed - as indicated by the VERSION2.zip that is included in my post

The original issue reported

Devorah_H_Intel — Fri, 07 Jul 2017 16:29:07 GMT

The original issue reported in this thread is fixed in the upcoming 18.0 compiler.

Thank you,

I think I just rediscovered

James_Pearson — Wed, 27 Sep 2017 20:51:26 GMT

I think I just spent a week or so rediscovering this bug in 2017 FORTRAN Update 4.

Devorah - can you verify the below example runs to completion using the new 18.0 compiler when compiled with the buffered IO optimization?

When I test with 2017 FORTRAN Update 4 and the buffered IO optimization it crashes while reading in the file it writes.

      program main
          IMPLICIT NONE
          
      integer :: scencqs = 53
      
      integer, pointer :: myreg(:), myk(:), myi(:), myj(:)
      real, pointer :: cq(:,:)
      real, pointer :: cvol(:), crhot(:)
      real, pointer :: cdt(:), ped(:), ued(:), ged(:), bed(:), sob(:)
      real, pointer :: pgd(:), pxd(:), ppd(:), pud(:)
      integer*4 :: lun,ic
      integer :: nctot = 0
      
      integer, pointer :: wmyreg(:), wmyk(:), wmyi(:), wmyj(:)
      real, pointer :: wcq(:,:)
      real, pointer :: wcvol(:), wcrhot(:)
      real, pointer :: wcdt(:), wped(:), wued(:),
     &                 wged(:), wbed(:), wsob(:)
      real, pointer :: wpgd(:), wpxd(:), wppd(:), wpud(:)
      integer*4 :: luno,wic
      integer :: wnctot
      
      print *, "Writing file..."
      
      luno=10
      open(unit=luno,form='unformatted',
     &file='test.input')
      
      wnctot=71445
      
      allocate(wmyreg(wnctot),wmyk(wnctot),wmyi(wnctot),wmyj(wnctot),
     &      wcq(scencqs,wnctot),wcvol(wnctot),wcrhot(wnctot),
     &      wcdt(wnctot),wged(wnctot),wbed(wnctot),
     &      wsob(wnctot),wped(wnctot),wued(wnctot))
      allocate(wpgd(wnctot),wpxd(wnctot),
     &      wppd(wnctot),wpud(wnctot))
      
      write(luno) (wmyreg(wic),wmyk(wic),wmyi(wic),wmyj(wic),
     &      wic=1,wnctot)
      write(luno) (wcq(1:scencqs,wic),wic=1,wnctot)
      write(luno) (wcvol(wic),wcrhot(wic),wcdt(wic),wged(wic),
     &      wbed(wic),wsob(wic),wped(wic),wued(wic),wic=1,wnctot)
      write(luno) (wpgd(wic),wpxd(wic),wppd(wic),wpud(wic),
     &      wic=1,wnctot)
      call flush(luno)
      close(luno)
          
      print *, "Reading file..."
            
      lun = 10
      open(unit=luno,form='unformatted',
     &file='test.input')
      
      nctot=71445
      
      allocate(myreg(nctot),myk(nctot),myi(nctot),myj(nctot),
     &      cq(scencqs,nctot),cvol(nctot),crhot(nctot),
     &      cdt(nctot),ged(nctot),bed(nctot),
     &      sob(nctot),ped(nctot),ued(nctot))
      allocate(pgd(nctot),pxd(nctot),
     &      ppd(nctot),pud(nctot))
      read(lun) (myreg(ic),myk(ic),myi(ic),myj(ic),ic=1,nctot)
      read(lun) (cq(1:scencqs,ic),ic=1,nctot)
      read(lun) (cvol(ic),crhot(ic),cdt(ic),ged(ic),
     &      bed(ic),sob(ic),ped(ic),ued(ic),ic=1,nctot)
      read(lun) (pgd(ic),pxd(ic),ppd(ic),
     &      pud(ic),ic=1,nctot)

      write(*,*) "Finished"
      close(lun)
      
      end program main

The code in #5 does not

mecej4 — Wed, 27 Sep 2017 23:21:00 GMT

The code in #5 does not follow either the fixed form or the free form of Fortran source. The compiler refuses to accept this nonstandard source form.

Hi James,

Devorah_H_Intel — Wed, 27 Sep 2017 23:32:09 GMT

Hi James,

Yes it builds and runs in 18.0

$ ifort -v
ifort version 18.0.0

$ ifort -g -traceback -assume buffered_io test2.f90

$ ./a.out

Writing file...
Reading file...
Finished

To add: the test fails at

Devorah_H_Intel — Wed, 27 Sep 2017 23:37:29 GMT

To add: the test fails at runtime on 17.0.4

My organization appears to

Craig_T_ — Fri, 13 Apr 2018 15:10:08 GMT

My organization appears to have been burned by this same problem using 17.0.6.270.

Devorah H., can you elucidate the scope of the problem fixed? Are there any situations where it's known that buffered and unformatted I/O using ifort2017 should still be safe?

Note my organization can not adopt ifort2018 because its run-time performance degraded for our code compared to ifort2017 for still unknown reasons.