Weird, but consistent behaviour of open statement when specifying a large recl (UPDATED))

[Todor_K_]

When I open a file for formatted output and write an array to it, for example with

open(123,file='proba.txt',form='formatted',recl=65535)
write(123,'10(1Xg0.14)') ARR

where ARR is an array of, say, 20 doubles, I just get plenty of blanks in file proba.txt.

I thought that recl modifies the maximum length of a record and my aim was to prevent ifort from breaking a record into multiple lines by setting this option to a huge value. Is my understanding of recl wrong, or is ifort exhibiting a buggy behavior? Although I have been most recently using ifort v17.0.2, I think i noticed the same "problem" with various older versions which is the reason for  `consistent' in the title.

If i set recl to a lower value, say 4000, it's all fine. As a side note, a) gfortran by default does not break longish record across lines and b) doesn't care about huge recls (maybe it ignores them?).

UPDATE after Steve's comment:

Sorry for the erroneous example. I was typing from memory, trying to reverse engineer a dynamic format statement. This time, I have isolated everything involved in my "line-oriented output of arrays" system.

module routines
  implicit none
  integer, parameter :: deflen = 65536
  interface write_line
     module procedure :: write_line_i
     module procedure :: write_line_r
    
  end interface write_line

contains
  subroutine write_line_i(fhandle, line)
    implicit none
    integer, intent(in) :: fhandle
    integer, contiguous, intent(in) :: line(:)
    character(len=*), parameter :: nfmt = 'I0'
    character(len=deflen) :: fmt
    write(fmt,'("(",I0,"(1X",A,"))")') size(line),nfmt
    fmt=adjustl(adjustr(fmt))
    write(fhandle,fmt) line

  end subroutine  write_line_i
  subroutine write_line_r(fhandle, line)
    implicit none
    integer, intent(in) :: fhandle
    real, contiguous, intent(in) :: line(:)
    character(len=*), parameter :: nfmt = 'g0.14'
    character(len=deflen) :: fmt
    write(fmt,'("(",I0,"(1X",A,"))")') size(line),nfmt
    fmt=adjustl(adjustr(fmt))
    write(fhandle,fmt) line

  end subroutine  write_line_r


end module routines
program writeproba
  use routines
  implicit none
  integer, parameter :: defrecl=65536
  integer, parameter :: N=12
  character(len=*),parameter :: header="thing1 thing2 thing3 thing4 thing5 thing6 thing7 thing8 thing8 thing10 thing11 thing12"
  real :: arr(N)
  integer :: i

  
  open(123,file='proba.txt',form='formatted',recl=defrecl)
  write(123,*) header
  do i=1,size(arr)
     arr(i)=i*acos(-1.)
  end do
  call write_line(123,arr)
  do i=1,size(arr)
     arr(i)=i**2*acos(-1.)
  end do
  call write_line(123,arr)
  close(123)
  
end program writeproba

This is compiled with

mpif90 -xHOST -integer-size 64 -real-size 64 -ip -ansi-alias -O3 writeproba.f90 -o writeproba

The program hangs if defrecl is 65536 and runs normally if set it to 4000. It turns out, this has nothing to do with the write_line routine, but with that line which writes out the header

write(123,*) header,

header being a character parameter.

[Steve_Lionel]

The code snippet you show can't possibly be representative as the format is syntactically incorrect. I fixed the errors and tried it in a complete program, but saw no problem.

The format you have (if correct) will start a new record every 10 elements - this behavior is specified by the Fortran standard. RECL= in Intel Fortran controls where list-directed output breaks lines, since that is not under your direct control. You aren't using list-directed.

If you think there is a real problem, please reply and include a short but complete program that demonstrates the issue.

[Todor_K_]

Is no one else getting this behavior? Isn't this a bug in a way? In my view, the compiler should at least warn that the recl is too big.

[mecej4]

Todor K. wrote:
 Is no one else getting this behavior? Isn't this a bug in a way? In my view, the compiler should at least warn that the recl is too big.

I think that your adding "updated/fixed" to the title may have caused potential readers to skip the thread, thinking that the issue had been taken care of!

I believe that there are compiler several bugs that are exposed by your test code. Taking one bug at a time and reporting it will have a better chance of the bugs being fixed. Here is the first one, which is related to output with G0.d format, and seems to be present in all versions of Ifort from 15.0.7.287 to the current version.

program ifortg0bug
implicit none
integer i
character(len=100) :: fmt = '(I2, 2x, 12(1Xg0.11))'
real, dimension(12) :: x = (/ (i, i=1,12) /)
print '(A)',trim(fmt)
print trim(fmt),1,x                 ! prints only asterisks and blanks
print '(I2, 2x, 12(1Xg0.11))',2,x
end program

The first PRINT statement displays the format, which is the same as the literal string in the third PRINT statement. Yet, the second PRINT statement causes asterisks and blanks to be printed out:

(I2, 2x, 12(1Xg0.11))
 1   *************** *************** *************** *************** *************** *************** *************** *************** *************** *************** *************** ***************
 2   1.0000000000 2.0000000000 3.0000000000 4.0000000000 5.0000000000 6.0000000000 7.0000000000 8.0000000000 9.0000000000 10.000000000 11.000000000 12.000000000

The bug goes away if g0.11 is replaced by g0.10, g0.9, etc.

[UPDATE, March 27, 2018] This bug has been reported to the developers, service request #: 03350148

[mecej4]

Here is the second compiler bug provoked by your test code. If a formatted file is opened with a RECL > 65534 (e.g., 65535 or 65636), and a character string is written to the file with a list-directed WRITE statement, the program appears to hang. If it is terminated with CTRL-C, we see that a very large file (many megabytes, instead of  the expected  29 bytes) has been produced, and contains nothing but millions of CR+LF pairs. You have already noted that it is the writing of the header that triggered the bug, but the following test code is simpler, and my observation regarding the contents of the output file should prove helpful.

program writeproba

  implicit none
!  
  open(123,file='proba.txt',form='formatted',recl=65536)
  write(123,*) 'thing1 thing2 thing3 thing4'
  print *,'Header written'
  close(123)
  
end program writeproba

[Todor_K_]

mecej4, thanks for confirming this! Yes, I noticed the asterisk characters, but only if I compiled the code with the default real size. With -real-size 64 everything was as expected. Probably I should have mentioned that, too. Good that you pointed this out.