Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

Bug with write and pos= in stream access

jacquet__olivier
Beginner
313 Views

Hello,

Up to now I was using the following version of the Intel Fortran compiler:
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 14.0.2.144 Build 20140120
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.

I am currently testing the last version of the Intel Fortran compiler:
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 17.0.1.132 Build 20161005
Copyright (C) 1985-2016 Intel Corporation.  All rights reserved.

I encounter what seems to be a bug when writing to a formatted file in Stream access and using the pos= functionality to write at different positions of the output file.

Here is a test program which behaves differently with the two versions of the compiler.

program TEST

    implicit none

    integer :: UNIT, I, POSITION1, POSITION2, IOK


    open(newunit=UNIT, file="test_stream", status="new", form="formatted", access="stream", iostat=IOK)
    write(6,*) IOK

    do I = 1, 2

       if (I == 1) then
          write(UNIT,"(A)") "a"
          inquire(unit=UNIT, pos=POSITION1)
       endif

       write(UNIT,"(I10)",pos=POSITION1) 2 * I

       if (I == 1) then
          write(UNIT,*)
          write(UNIT,"(A)") "a"
          inquire(unit=UNIT, pos=POSITION2)
       endif

       write(UNIT,"(I10)",pos=POSITION2) 2 * I + 1

    enddo

end program TEST

The expected output is what I get with the older version:

a
         4
 
a
         5

Here is what I get whith the last version:

a
         4
^@^@^@^@         5

Best regards.

Olivier

 

0 Kudos
7 Replies
Kevin_D_Intel
Employee
313 Views

I reproduced this. It appears the underlying defect was introduced in our 16.0 release. I escalated it to Development. Thank you for the convenient reproducer.

(Internal tracking id: DPD200416585)

(Resolution Update on 12/14/2016): Closed. Not a defect. Refer to subsequent reply)

0 Kudos
Kevin_D_Intel
Employee
313 Views

Turns out this not a defect. Development clarified the output from 17.0 (and 16.0) compilers is correct and Standard conformant, and that output from pre-16.0 compilers was not. They referred me to the Fortran 2008 Standard, where in 9.3.4.4 paragraph 6 it says:

    "For a formatted stream output statement, if no error condition occurred, the terminal point of the file is set to
     the next position after the highest-numbered position to which data was transferred by the statement."

(This is also in the F2003 Standard, in section 9.2.3.3, lines 30-31.)

They note other text in the Standard implies that unformatted stream writes do not truncate and work as you were expecting.

They also provided the following walk-through for your program to help explain the output file contents with 16.0 and newer compilers that I wanted to also share. They wrote:

The sequence of events in this program is:

 === Initially at position 1
 === Was at  1, wrote          "a\n" at position  1, now at position  3, save 3 as POSITION1
 === Was at  3, wrote "         2\n" at position  3, now at position 14, using POSITION1
 === Was at 14, wrote          " \n" at position 14, now at position 16
 === Was at 16, wrote          "a\n" at position 16, now at position 18, save as POSITION2
 === Was at 18, wrote "         3\n" at position 18, now at position 29
 === Was at 29, wrote "         4\n" at position  3, now at position 14, using POSITION1, truncating file (removing from 14 to 29)
 === Was at 14, wrote "         5\n" at position 18, now at position 29, using POSITION2 and extending file with NULL-character fill.
 === Finally at position           29

I hope that helps explain the different behavior between the compilers. Thanks for posting this. It was insightful.

0 Kudos
jacquet__olivier
Beginner
313 Views

Thank you but it's too bad if there is no solution to do what I could do thanks to this anomaly, isn't it?

Do you have any suggestion?

Olivier

0 Kudos
Kevin_D_Intel
Employee
313 Views

I inquired (via a feature request) about the previous functionality being supported as an extension and will let you know what I hear about that.

(Internal tracking id: DPD200416652 - Feature request)

If you want/need to do formatted output and store it in a STREAM file but keep the random positioning, then one Developer solution is to do the output in two steps:

1.      write formatted output to a CHARACTER(<n>) variable, where the size <n> is big enough to hold the result;
2.      write TRIM(<variable>) to an UNFORMATTED STREAM;

and use quotes or new-lines to delimit parts for later reading if required.  Their example code is below.

program TEST

    implicit none
    integer :: UNIT, I, POSITION1, POSITION2, here, where, IOK
    CHARACTER*200  line

    ! UNFORMATTED STREAM file.
    ! The file can be positioned because it is STREAM.
    ! This is a way to do direct-access writes to a file which can then
    ! be read as formatted records.
    !
    open(newunit=UNIT, file="work_around.txt", status="replace", form="unformatted", access="stream", iostat=IOK)

    inquire(unit=UNIT, pos=where)
    print *,'=== Initally at position ',where

    do I = 1, 2
        if (I == 1) then
            !
            ! Writes "a\n" at positions 1,2; 3 is next and saved as
            ! POSITION1.
            !
            write(line,"(2A)") "a", CHAR(10)
            write(UNIT) TRIM(line)
            inquire(unit=UNIT, pos=POSITION1)
            print *,'=== Wrote "a" at position',where,', now at position', position1
        endif

        ! i == 1: Writes "        2\n" at positions 3..13; 14 is next
        ! i == 2: Writes "        4\n" at positions 3..13; 14 is next
        !
        ! The i==2 write truncates (fills buffer with NULL characters)
        ! from position 14 to end.
        inquire(unit=UNIT, pos=here)
        !
        write(line,"(I10,A)") 2 * I, CHAR(10)
        write(UNIT,pos=POSITION1) TRIM(line)

        inquire(unit=UNIT, pos=where)
        print *,'=== Was at',here,', wrote',2*i,' at position ', position1,', now at position',where

        if (I == 1) then
            !
            ! Writes " \n" at positions 14,15; 16 is next
            !
            inquire(unit=UNIT, pos=here)
            write(line,*) CHAR(10)
            write(UNIT) TRIM(line)
            inquire(unit=UNIT, pos=where)
            print *,'=== Wrote empty at ',here,', now at position',where

            ! Writes "a\n" at positions 16,17; 18 is next
            !
            ! Save value 18 as POSITION2.
            !
            here = where
            write(line,"(2A)") "a", CHAR(10)
            write(UNIT) TRIM(line)
            inquire(unit=UNIT, pos=POSITION2)
            print *,'=== Wrote "a" at',here,', now at',POSITION2,' saving it as POSITION2'
         endif

        ! i == 1: Writes "        3\n" at positions 18..28; 29 is next
        ! i == 2: Writes "        5\n" at positions 18..28; 29 is next
        !
        ! The i==2 write truncates writes at saved position 18.
        ! past the nulls in 14..17 from the previous 1==2 write.
        !
        inquire(unit=UNIT, pos=here)
        write(line,"(I10,A)") 2 * I + 1, CHAR(10)
        write(UNIT,pos=POSITION2) TRIM(line)
        inquire(unit=UNIT, pos=where)
        print *,'=== Was at',here,', wrote',2*i+1,' at saved position',POSITION2,', now at', where
    enddo

    ! At 29.
    !
    inquire(unit=UNIT, pos=where)
    print *,'=== Finally at position ',where
    close(UNIT)

end program TEST

 

0 Kudos
mecej4
Honored Contributor III
313 Views

Instructive, but also troubling. First, however, an errata for #3, "sequence of events". The third event will only write a newline, so the count should increase by only 1, to 15, rather than 16, unless the run was made under Windows; this correction then propagates to the rest of the position numbers.

What is troubling is that, after the file has been truncated, with the last logical byte of the file at byte 13 one can merrily write anywhere beyond that point, making the concepts of end of file and current file length meaningless. If, furthermore, the compiler does not check to see if the value used in POS=<value> was one of those that was the result of an INQUIRE(..,POS=<value>), as the Fortran standard requires, my program could do a lot of damage unless the OS or Fortran/C RTL steps in to limit the damage.

The standard states that the implementation may place further restrictions on how stream files are handled. For example, it allows the implementation to decide whether or not WRITE(...,pos=<v>) is allowed for a stream file. Presumably, IFort may allow this, and another Fortran compiler may disallow it. What troubles me is that I see no way of determining whether or not such access is allowed by the current compiler. A related dilemma: the standard states that the variable v in an INQUIRE(..,POS=v) can become undefined if a file has characteristics that disallow the determination of the position. However, there is no way (as far as I can see) to determine beforehand (by using INQUIRE, etc.) if POS= is supported for that file, or to determine whether v has become undefined after an improper inquiry. In other words, there is no "POSITIONABLE=CanPosition" clause in the INQUIRE statement.

Please note that there is no urgency at all. I rarely get deep into complex I/O, and I would probably trust the simpler stream I/O model of C if I need to do stream I/O. But, one of these days, we may need to know.

0 Kudos
Kevin_D_Intel
Employee
313 Views

Thank you. I shared your observations w/Development.

0 Kudos
mecej4
Honored Contributor III
313 Views

Here are some additional twists to this complicated I/O issue. If the program of #1 is run a second time, seemingly strange things happen:

          10
forrtl: severe (110): stream data transfer statement is not allowed to this unit, unit 3077468, file s:\LANG\
Image              PC        Routine            Line        Source
libifcoremd.dll    63B91D9F  Unknown               Unknown  Unknown
libifcoremd.dll    63C0495D  Unknown               Unknown  Unknown
xio.exe            0034119A  _MAIN__                    17  xio.f90
xio.exe            003430DF  Unknown               Unknown  Unknown

The OPEN failed because STATUS='NEW' is specified and the file was already created in the first run. Because a NEWUNIT clause is used, as per the Fortran standard the file unit no., UNIT, retains the same value as it did before the failed OPEN. The way the program is written, UNIT was expected by the author to be defined after the OPEN statement, but in this case it was undefined at program start, so it stays undefined. From this point on, doing INQUIRE on this unit number (3077468, 2947932, etc.) is impossible because these garbage values may be well beyond the range of allowed unit numbers. Better assign a known value to UNIT before calling OPEN (NEWUNIT=UNIT,..).

How did the file name become fort.3077468, when the OPEN specified test_stream? The OPEN failed, so not only is UNIT now undefined, but the file is not connected. The program is next going to attempt to WRITE to an unconnected file with an invalid unit number. Intel Fortran does not seem to mind a unit number of 3077468 and the WRITE statement, as the standard prescribes, does an implicit OPEN of fort.3077468 as an ordinary sequential formatted file (no keyword='string' clauses in this implicit OPEN), and then attempts to do stream I/O to it.

All this can be quite difficult to anticipate and debug. Therefore, a good measure of defensive programming may be needed if one uses the convenience of NEWUNIT and the flexibility of stream I/O. It was after I added an IOMSG=msg clause to the I/O statements and displayed msg that I understood what was happening.

0 Kudos
Reply