Up to now I was using the following version of the Intel Fortran compiler:
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 22.214.171.124 Build 20140120
Copyright (C) 1985-2014 Intel Corporation. All rights reserved.
I am currently testing the last version of the Intel Fortran compiler:
Intel(R) Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 126.96.36.199 Build 20161005
Copyright (C) 1985-2016 Intel Corporation. All rights reserved.
I encounter what seems to be a bug when writing to a formatted file in Stream access and using the pos= functionality to write at different positions of the output file.
Here is a test program which behaves differently with the two versions of the compiler.
integer :: UNIT, I, POSITION1, POSITION2, IOK
open(newunit=UNIT, file="test_stream", status="new", form="formatted", access="stream", iostat=IOK)
do I = 1, 2
if (I == 1) then
write(UNIT,"(I10)",pos=POSITION1) 2 * I
if (I == 1) then
write(UNIT,"(I10)",pos=POSITION2) 2 * I + 1
end program TEST
The expected output is what I get with the older version:
Here is what I get whith the last version:
I reproduced this. It appears the underlying defect was introduced in our 16.0 release. I escalated it to Development. Thank you for the convenient reproducer.
(Internal tracking id: DPD200416585)
(Resolution Update on 12/14/2016): Closed. Not a defect. Refer to subsequent reply)
Turns out this not a defect. Development clarified the output from 17.0 (and 16.0) compilers is correct and Standard conformant, and that output from pre-16.0 compilers was not. They referred me to the Fortran 2008 Standard, where in 188.8.131.52 paragraph 6 it says:
"For a formatted stream output statement, if no error condition occurred, the terminal point of the file is set to
the next position after the highest-numbered position to which data was transferred by the statement."
(This is also in the F2003 Standard, in section 184.108.40.206, lines 30-31.)
They note other text in the Standard implies that unformatted stream writes do not truncate and work as you were expecting.
They also provided the following walk-through for your program to help explain the output file contents with 16.0 and newer compilers that I wanted to also share. They wrote:
The sequence of events in this program is:
=== Initially at position 1
=== Was at 1, wrote "a\n" at position 1, now at position 3, save 3 as POSITION1
=== Was at 3, wrote " 2\n" at position 3, now at position 14, using POSITION1
=== Was at 14, wrote " \n" at position 14, now at position 16
=== Was at 16, wrote "a\n" at position 16, now at position 18, save as POSITION2
=== Was at 18, wrote " 3\n" at position 18, now at position 29
=== Was at 29, wrote " 4\n" at position 3, now at position 14, using POSITION1, truncating file (removing from 14 to 29)
=== Was at 14, wrote " 5\n" at position 18, now at position 29, using POSITION2 and extending file with NULL-character fill.
=== Finally at position 29
I hope that helps explain the different behavior between the compilers. Thanks for posting this. It was insightful.
I inquired (via a feature request) about the previous functionality being supported as an extension and will let you know what I hear about that.
(Internal tracking id: DPD200416652 - Feature request)
If you want/need to do formatted output and store it in a STREAM file but keep the random positioning, then one Developer solution is to do the output in two steps:
1. write formatted output to a CHARACTER(<n>) variable, where the size <n> is big enough to hold the result;
2. write TRIM(<variable>) to an UNFORMATTED STREAM;
and use quotes or new-lines to delimit parts for later reading if required. Their example code is below.
program TEST implicit none integer :: UNIT, I, POSITION1, POSITION2, here, where, IOK CHARACTER*200 line ! UNFORMATTED STREAM file. ! The file can be positioned because it is STREAM. ! This is a way to do direct-access writes to a file which can then ! be read as formatted records. ! open(newunit=UNIT, file="work_around.txt", status="replace", form="unformatted", access="stream", iostat=IOK) inquire(unit=UNIT, pos=where) print *,'=== Initally at position ',where do I = 1, 2 if (I == 1) then ! ! Writes "a\n" at positions 1,2; 3 is next and saved as ! POSITION1. ! write(line,"(2A)") "a", CHAR(10) write(UNIT) TRIM(line) inquire(unit=UNIT, pos=POSITION1) print *,'=== Wrote "a" at position',where,', now at position', position1 endif ! i == 1: Writes " 2\n" at positions 3..13; 14 is next ! i == 2: Writes " 4\n" at positions 3..13; 14 is next ! ! The i==2 write truncates (fills buffer with NULL characters) ! from position 14 to end. inquire(unit=UNIT, pos=here) ! write(line,"(I10,A)") 2 * I, CHAR(10) write(UNIT,pos=POSITION1) TRIM(line) inquire(unit=UNIT, pos=where) print *,'=== Was at',here,', wrote',2*i,' at position ', position1,', now at position',where if (I == 1) then ! ! Writes " \n" at positions 14,15; 16 is next ! inquire(unit=UNIT, pos=here) write(line,*) CHAR(10) write(UNIT) TRIM(line) inquire(unit=UNIT, pos=where) print *,'=== Wrote empty at ',here,', now at position',where ! Writes "a\n" at positions 16,17; 18 is next ! ! Save value 18 as POSITION2. ! here = where write(line,"(2A)") "a", CHAR(10) write(UNIT) TRIM(line) inquire(unit=UNIT, pos=POSITION2) print *,'=== Wrote "a" at',here,', now at',POSITION2,' saving it as POSITION2' endif ! i == 1: Writes " 3\n" at positions 18..28; 29 is next ! i == 2: Writes " 5\n" at positions 18..28; 29 is next ! ! The i==2 write truncates writes at saved position 18. ! past the nulls in 14..17 from the previous 1==2 write. ! inquire(unit=UNIT, pos=here) write(line,"(I10,A)") 2 * I + 1, CHAR(10) write(UNIT,pos=POSITION2) TRIM(line) inquire(unit=UNIT, pos=where) print *,'=== Was at',here,', wrote',2*i+1,' at saved position',POSITION2,', now at', where enddo ! At 29. ! inquire(unit=UNIT, pos=where) print *,'=== Finally at position ',where close(UNIT) end program TEST
Instructive, but also troubling. First, however, an errata for #3, "sequence of events". The third event will only write a newline, so the count should increase by only 1, to 15, rather than 16, unless the run was made under Windows; this correction then propagates to the rest of the position numbers.
What is troubling is that, after the file has been truncated, with the last logical byte of the file at byte 13 one can merrily write anywhere beyond that point, making the concepts of end of file and current file length meaningless. If, furthermore, the compiler does not check to see if the value used in POS=<value> was one of those that was the result of an INQUIRE(..,POS=<value>), as the Fortran standard requires, my program could do a lot of damage unless the OS or Fortran/C RTL steps in to limit the damage.
The standard states that the implementation may place further restrictions on how stream files are handled. For example, it allows the implementation to decide whether or not WRITE(...,pos=<v>) is allowed for a stream file. Presumably, IFort may allow this, and another Fortran compiler may disallow it. What troubles me is that I see no way of determining whether or not such access is allowed by the current compiler. A related dilemma: the standard states that the variable v in an INQUIRE(..,POS=v) can become undefined if a file has characteristics that disallow the determination of the position. However, there is no way (as far as I can see) to determine beforehand (by using INQUIRE, etc.) if POS= is supported for that file, or to determine whether v has become undefined after an improper inquiry. In other words, there is no "POSITIONABLE=CanPosition" clause in the INQUIRE statement.
Please note that there is no urgency at all. I rarely get deep into complex I/O, and I would probably trust the simpler stream I/O model of C if I need to do stream I/O. But, one of these days, we may need to know.
Here are some additional twists to this complicated I/O issue. If the program of #1 is run a second time, seemingly strange things happen:
10 forrtl: severe (110): stream data transfer statement is not allowed to this unit, unit 3077468, file s:\LANG\ Image PC Routine Line Source libifcoremd.dll 63B91D9F Unknown Unknown Unknown libifcoremd.dll 63C0495D Unknown Unknown Unknown xio.exe 0034119A _MAIN__ 17 xio.f90 xio.exe 003430DF Unknown Unknown Unknown
The OPEN failed because STATUS='NEW' is specified and the file was already created in the first run. Because a NEWUNIT clause is used, as per the Fortran standard the file unit no., UNIT, retains the same value as it did before the failed OPEN. The way the program is written, UNIT was expected by the author to be defined after the OPEN statement, but in this case it was undefined at program start, so it stays undefined. From this point on, doing INQUIRE on this unit number (3077468, 2947932, etc.) is impossible because these garbage values may be well beyond the range of allowed unit numbers. Better assign a known value to UNIT before calling OPEN (NEWUNIT=UNIT,..).
How did the file name become fort.3077468, when the OPEN specified test_stream? The OPEN failed, so not only is UNIT now undefined, but the file is not connected. The program is next going to attempt to WRITE to an unconnected file with an invalid unit number. Intel Fortran does not seem to mind a unit number of 3077468 and the WRITE statement, as the standard prescribes, does an implicit OPEN of fort.3077468 as an ordinary sequential formatted file (no keyword='string' clauses in this implicit OPEN), and then attempts to do stream I/O to it.
All this can be quite difficult to anticipate and debug. Therefore, a good measure of defensive programming may be needed if one uses the convenience of NEWUNIT and the flexibility of stream I/O. It was after I added an IOMSG=msg clause to the I/O statements and displayed msg that I understood what was happening.