Difficulty reading Direct access file

[WSinc]

I want to read a file which was NOT created by the fortran library.

So in order to manipulate bytes within it, I want to open it with something like:
OPEN(3,access="direct",recl=64,file='test.dat', etc.

So I can read the file in with the 64 words of 4 bytes each OK, but
the problemoccurs when I get to the last record.

If the file does not have a size exactly a multiple of 256 bytes, then I get a error of 36 when reading the last part of the file. Apparently the Fortran library does not properly handle a situation when I have part of a record left over. Ir expects an exact number of records of the same size.

The no of bytes in the I/O input list does not make any difference. Example:
Suppose the file has a length of 2600 bytes. Then that would be 10 records with 40 bytes left over.
10*256 + 40 = 2600

So I can't read in the last 40 bytes even though the I/O list is 40 bytes long, i.e. 10 integer*4 quantities, or 5 real*8 numbers.

Is there a way around this? Apparently knowing the exact size of the file does not help.

[IanH]

Consider using STREAM instead of DIRECT. It is a F2003 feature, which regards the file as a stream of "file storage units" (which I think is bytes by default with ifort).

If you are reading arbitrary records (i.e. you are not just starting at the beginning of the file and reading through to the end) you will have to deal with the positioning yourself by using the POS specifier in the READ statements, but that's not hard (POS = (record_number-1) * record_size_in_bytes + 1).

[WSinc]

That sounds like a possibility. The file needs to be regarded as a set of continuous bytes with no predetermined file or record structure. When it reaches the end of file, does it give an EOF return?

I'll play around with it - thanks for your tip.

[dboggs]

When you say "not created by the fortran library" do you mean that it is not a fortran file? How do you even know the file is divided into records? A Fortran "unformatted" file is divided into records separated by fortran delimeters.

I would consider opening it as "binary". This has no internal structure; there are no such things as "records". You just read bytes (or words) until you get to the end.

Unfortunately binary file handling is not an official Fortran feature but an IVF extended feature, I believe. But it's useful and we use it frequently.

[IanH]

Stream access imposes no file or record structure on the file. It is the standardised variant of "binary". Stream access is reasonably well supported in compilers that are heading towards F2003 support.

You should be able to get an end-of-file condition via the IOSTAT specifier in a read statement.

(An unformatted fixed length record file generated (and hence read) by ifort does not have record markers by default, but that is an implementation detail.)

[WSinc]

Hi -

I played around with it some, and found something weird.......

For quantities longer than 1 byte, it writes them outin REVERSE ORDER.
That is, least signicant part first.

I am wondering if there is a wayto change that.

For example if I start out with POS=1 and write out an integer*4 number,
the first byte(=1) has the least significant part, and byte 4 has the most significant part.

There is hardly any documentation about this topic, which is surprising considering how
long the standard has been implemented.

It appears that the longer quantities are also read back in in reverse order as well.

Apparently this is in direct contradiction to the way Fortran I/O usually orders things.

[IanH]

The standard doesn't specify how an integer (or any other type) is represented in the file - different architectures might represent integers in different ways in memory (they might not even be binary!), so for efficiency's sake they have the freedom to use that different representation in any unformatted files that they write.

The little-endian nature (least signficant byte first) of the data in the file matches the way that integers are laid out in memory for the "intel compatible" CPU that you are using. Unless you've got some special command line options (/convert:xxx, in the VS environment the propery is under "Compatibility" > "Unformatted File Conversion") or specifiers in your open statements, this is the same way that data is written with any unformatted output.

There are topics in the compiler documentation under "Compiler Reference" > "Data and I/O" > "Data Representation" that have a fair bit of information.

If you described your goals we might be able to suggest a suitable approach. (For example - if you just want to read bytes, then use a byte sized integer.)

[WSinc]

I probably will use Interger*1 or BYTE to read the quantities, then just parse them
individually, as you suggested.

So you are saying the I can't necessarily read in a file from another platform and expect the
quantities to be stored the same?

I will check the documentation further about that. But I think they need to go into much more detail regarding the STREAM option. I will also experiment further with unformatted Fortran files.

Thanks for your input.

Yours; Bill S.

[tropfen]

Hello,

working in the world of climate modelers (including working on more then one platform), if have learnt that:

convert="BIG_ENDIAN" or similar is a MUST.

What also includes the knowledge about who has produced the file on which platform. The best way is to include this information in the filenames.

Frank