- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I was experimenting with different ways to read a CSV file and found that this program terminates on
a rather unconspicuous location in the file:
! checkstream.f90 --
! Read a CSV file, using stream access
!
program checkstream
implicit none
integer :: i, k, kn, count
integer :: ierr
integer :: start, stop, clock_rate
real :: x, y, z, sum
character(len=20) :: xstr, ystr, zstr, string
!
! Read the file as a stream
! NO success with Intel Fortran 11.0
!
do i = 1,2
open( 10, file = 'input.csv', access = 'stream', form = 'formatted' )
count = 0
call system_clock( start, clock_rate )
do
read( 10, *, iostat = ierr ) x, y, z, string
! read( 10, * ) x, y, z, string
! write( *, * ) count, x, y, z, string
if ( ierr /=0 ) then
exit
endif
count = count + 1
sum = sum + x + y + z
enddo
call system_clock( stop )
close( 10 )
enddo
write(*,*) 'Method 4 (formatted stream):', (stop-start)/real(clock_rate), count
end program checkstream
I have generated the file with this program:
! createfile.f90 --
! Program to generate a large CSV file
!
program createfile
implicit none
real :: x, y, z
character(len=20) :: string
integer :: i
integer :: k
open( 10, file = 'input.csv' )
string = 'abcdefghijklmnopqrst'
do i = 1,1000000
call random( x )
y = 1.0 - x
z = 1000.0 * x
k = min(20, int(1 + 20.0 * x) )
write( 10, '(f0.3,a,f0.5,a,f0.2,a,a)' ) x, ',', y, ',', z, ',', string(1:k)
enddo
end program createfile
and using Intel Fortran 11.0 (on either Windows XP or Linux) the first program simply stops
after several thousand lines.
Here is a part of the output:
2384 0.5490000 0.4509600 549.0400 abcdefghijk
2385 0.4890000 0.5111500 488.8500 abcdefghij
2386 0.1370000 0.8629200 137.0800 abc
2387 0.9820000 1.8090000E-02 981.9100 abcdefghijklmnopqrst
2388 0.6190000 0.3813700 618.6300 abcdefghijklm
2389 0.7670000 0.2331600 766.8400 abcdefghijklmnop
forrtl: severe (59): list-directed I/O syntax error, unit 10, file .../input.csv
Image PC Routine Line Source
checkstream 080996FD Unknown Unknown Unknown
checkstream 08098C75 Unknown Unknown Unknown
checkstream 0807ACF8 Unknown Unknown Unknown
checkstream 0806CF03 Unknown Unknown Unknown
checkstream 0806C860 Unknown Unknown Unknown
checkstream 08055869 Unknown Unknown Unknown
checkstream 08049F5C Unknown Unknown Unknown
checkstream 08049E01 Unknown Unknown Unknown
libc.so.6 00144E8C Unknown Unknown Unknown
checkstream 08049CF1 Unknown Unknown Unknown
The line it is apparently trying to read looks like:
0.250,0.74994,250.06,abcdef
I can not see anything wrong with that line or the program.
Can anyone shed some light on this?
Regards,
Arjen
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
do i = 1,2 is a left-over from a larger program.
I read the file twice to exclude caching effects in the timings.
I have some further information - reading the line into a string
and then reading the data from that string enabled me to print
what exactly is being read from the file.
It turns out that the line read from the file is incomplete:
994,250.06,abcdef
instead of:
0.250,0.74994,250.06,abcdef
(the first 10 characters aremissing)
That explains why the program stops with this error, but the new puzzle is: why
does it read only a part of the line?
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
decides to skip some characters. Really puzzling!
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Under Linux I have version 11.0-1569, as reported by the -what option (though
from the installation directory structure I would have thought a version 11.0.081 would have been more appropriate).
Under Windows I have 11.0.072, so I guess that is slightly older than 11.0.074.
Perhaps it is time to ask my IT department to upgrade.
Regards,
Arjen
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I am not sure that ACCESS='STREAM' and FORM='FORMATTED' are compatible. Stream, to me, implies a raw data output to mimic the binary mode in the C language. In this mode, there is no file formatting ( CR-LF for example, as you have in this common separated file). I can see where the runtime is getting confused and reading CR LF at the line ends.
I'll open a bug report since there may be some bug with STREAM. But to me, I would disallow the open with ACCESS=STREAM and FORM=FORMATTED - these 2 are IMHO incompatible.
ron
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page