XE12--> XE13 : performance issue when reading files

[Guillaume_d_]

Hello, 

  I have a program that read quite big files (~ 100 Mo). Reading time was not a problem until I updated to XE 13. Below is a part of the program that is now very slow. 

XE12 --> 0.03 s  (clock time)

XE13 --> 5.50 s  (clock time)  , whatever the update  (13.0 , 13.1.1 ... )

Regards, 

   Guillaume

call chrono2%start()

do ihdg=1, nbhead

   do k=1, nksym
      read(9)(ptvxyz3(k,ift,ihdg),ift=1,nfhull+nfzero+nlwlin+nwlzero+nfboit+nwboit)
   end do
   do k=1, 2**nsymhull(1)
      read(91)(ptvxyz_zz3(k,ift,ihdg),ift=1,nfhull+nfzero+nlwlin+nwlzero+nfboit+nwboit)
   end do
   read(9)( CmDif3(ib,ihdg),ib=0,NBES )
   read(9)( SmDif3(ib,ihdg),ib=0,NBES )
end do

call chrono2%stop()

[Steven_L_Intel1]

Perhaps this is covered by the following release note?

3.2.7 Change in File Buffering Behavior (13.1)
In product versions prior to Intel® Visual Fortran Composer XE 2013 (compiler version 13.0),
the Fortran Runtime Library buffered all input when reading variable length, unformatted
sequential file records. This default buffering was accomplished by the Fortran Runtime Library
allocating an internal buffer large enough to hold any sized, variable length record in memory.

For extremely large records this could result in an excessive use of memory, and in the worst
cases could result in available memory being exhausted. The user had no ability to change this
default buffering behavior on such READs. There was always the ability to request or deny
buffering of these records when writing them, but not when reading them.

This default buffering behavior was changed with the release of Intel® Visual Fortran Composer
XE 2013. Beginning with this version, all such records are not buffered by default, but rather
read directly from disk to the user program’s variables. This change helped programs that
needed to conserve memory, but could in fact result in a performance degradation when
reading records that are made of many small components. Some users have reported this
performance degradation.

The Intel® Visual Fortran Composer XE 2013 Update 2 (compiler version 13.1) release of the
Fortran Runtime Library now provides a method for a user to choose whether or not to buffer
these variable length, unformatted records. The default behavior remains as it was in 13.0;
these records are not buffered by default. If you experience performance degradation when
using 13.1 with this type of I/O, you can enable buffering of the input the same way that you
choose whether to enable buffering of the output of these records – one of the following:

- specifying BUFFERED=”YES” on the file's OPEN statement
- specifying the environment variable FORT_BUFFERED to be YES, TRUE or an
integer value greater than 0
- specifying -assume buffered_io on the compiler command line

In the past, these mechanisms applied only when issuing a WRITE of variable length,
unformatted, sequential files. They can now be used to request that the Fortran Runtime
Library buffer all input records from such files, regardless of the size of the records in the file.

Using these mechanisms returns the READing of such records to the pre-13.0 behavior.

[Guillaume_d_]

Thanks Steve, I had already read that part and already switch on  "-assume buffered_io" in my program. However, I have just had a closer look and it appears the file that I read was opened in a library where I forgot to use this switch. Using "-assume buffered_io" in the library that open the file solve the issue.

Regards, 

  Guillaume

[Steven_L_Intel1]

Glad to hear it.