Using RECL in an INQUIRE statement on Unformatted File

[efengle]

I am having trouble getting the correct record size of a record in a file that I have opened as FORM="UNFORMATTED".  Here is a code snippet.  The file I am reading is a big-endian file.

OPEN(UNIT=20,FILE="FILENAME",FORM="UNFORMATTED",CONVERT="BIG_ENDIAN",STATUS="OLD")

INQUIRE(UNIT=20,RECL=RECSIZE)
PRIINT *, RECSIZE

I would expect RECSIZE to be the number of bytes (in 4-byte units) of the next record, which is the first record in this case.  Am I misinterpreting the use of RECL specifier in an INQUIRE statement?  In my example, I am expecting a record length of 5840 bytes (or 1460 4-byte units).  Instead, RECSIZE=510?

Any help/info is much appreciated.

[Steven_L_Intel1]

Your file has no fixed record length - it has variable length records.  What I think you're getting is the blocksize.

[efengle]

OK.  The value of 510 make sense.  After I get RECSIZE, I perform RECSIZE=RECSIZE-2 (sorry for not explaining that part).  So the typical 512 block size would give me 510.  Is there a way to get the record length in a UNFORMATTED file prior to reading the actual record?  My goal in this is to read from the record into an array (4-byte INTEGER) that has been ALLOCATED for the exact amount of the record length.

[jimdempseyatthecove]

Open the file in stream mode. Then

read the record length from the stream, allocate buffer, read the data, read the ending record length of prior record.

Note, it is your responsibility to figure out the number of bytes in the record length.

*** do not assume that because you see 4 bytes today that you will see 4 bytes tomorrow or on a different platform ***

IOW: Open file in unformatted mode, read first record to obtain record size of first record, use GETPOS to get the position in the file following the first record. The return value from GETPOS - record length should equal 2x the size of the number of bytes holding the record length. *** do not assume this will always be 4 bytes or always be the same on all systems or always be the same endian-ness. close file, open file in stream mode, read the prior returned GETPOS number of bytes into integer(1) array. Verify the two sentinels of record size, which may be 1, 2, 4, 8 bytes are indeed at positions assumed and of size assumed. If not, you have some more homework to do. e.g. record size bytes may be big-endian or GETPOS returning record number not byte offset.

If the record size values (both of them) are in the positions assumed and endian-ness assumed, condition the program to use the number of bytes discovered that hold the record length and endian-ness. Note, records are stored:

{recordLength}data{recordLength}
{recordLength}dataMayBeDifferentLength{recordLength}

Jim Dempsey

[Timothy_P_Intel]

If the file wasn't created by an ifort compatible compiler, access='stream' might be the way to go.  Several popular linux Fortran compilers should be compatible with ifort unformatted files.

Originally, I thought your issue might be about the legacy default of ifort; if you want recl to be measured in bytes, you must set -assume byterecl.  recl applies only to files created and opened with access='direct'

[jimdempseyatthecove]

Then if recl= does not apply to unformatted files then you can still open in unformatted, read first record into "large" buffer (with error test), use GETPOS to get the byte position after the first record. Close the file, open in stream and then read into the integer(1) array of size found by GETPOS. Then by inspection of first few and last few bytes you can figure out the number of bytes per record size. By the way, this new forum comment section really sucks. The designer must think we tweet these comments with cell phones. Jim Dempsey

[efengle]

I left out some information in my original post to keep it short since I am working with an in-house scientific data compression format.  Our in-house data format, called TDLPACK is written to Fortran unformatted sequential files in the following manner:

<recordlength>{4-byte padding}{size of TDLPACK record in bytes}{TDLPACK data}<recordlength>

So here is the solution I came up with,

OPEN(UNIT=20,FILE="FILENAME",FORM="UNFORMATTED",CONVERT="BIG_ENDIAN",STATUS="OLD)

READ(20)NTRASH,IOCTET
ALLOCATE(IPACK(IOCTET/4))
BACKSPACE(20)
READ(20)NTRASH,IOCTET,IPACK

where NTRASH is the 4-byte padding, IOCTET is the TDLPACK size in bytes, and IPACK is the integer array holding the TDLPACK record.  The code above assumes 4-byte INTEGER as the default.

Does anyone see an issue with this?  My reasoning for originally posting was to see if there was a way to get the record length without any explicit I/O...hoping the RECL specifier in the INQUIRE statement would be the solution.

Thanks for all feedback!

[jimdempseyatthecove]

If you opened in stream mode .and. made the assumption that the hidden record lengths were 4 bytes, you could stream read three 4-byte fields: Fortran record length, padd, size of TDLPACK Then allocate your IPACK, stream read the data, .and. read the trailing hidden record length. This would remove the BACKSPACE requirement, thus improving performance. The suggestion I made is no different from above for reading the stream. The procedural difference is at file open time, I suggest inserting some sanity check code that determines/verifies what the byte size and endian-ness is of the hidden Fortran record length. Also note that implicit with the endian-ness of the discovered Fortran record size, .and. discovered endian-ness of your size of TDLPACK record, you derive the endian-ness of the data. What this does for you is your files are transportable without regard to endian-ness of recorded data and curiously you can also have your code figure out the number of bytes for TDLPACK record size. Think of someone giving you an image file of a tape from an old system (byte stream). If you write the code well, you will not have to spend time later figuring out the data layout. Jim Dempsey