READ for REC=1 gives different results

[sindizzy]

I am trying to convert a program from Lahey Fortran to Intel IFORT. One routine is giving me headaches. When I compile the code below in Lahey I get RCNT=13419. which is what is expected. I now put the same code in VS and compile with IFORT and I get RCNT= 0.

I don't understand my defect. Is there a switch that I am missing that in the past I may have taken for granted?

       PROGRAM NEWCNT

       IMPLICIT NONE

       CHARACTER * 255  BinFile, OutFile

       REAL        RCNT

      BinFile='C:\docs\CITY.BIN'
      OPEN (UNIT=7, FILE=BinFile, ACCESS='DIRECT', RECL=22000,
+ STATUS='OLD', FORM='UNFORMATTED')

      READ (7, REC=1) RCNT

      WRITE (*, '(A,F6.0)') 'RCNT=', RCNT

       CLOSE(7)

       END PROGRAM NEWCNT

[Steve_Lionel]

Try /assume:byterecl. Intel Fortran defaults unformatted RECL to 4-byte units, Lahey does not. /standard-semantics will enable this and other options, but that's a bigger hammer.

[sindizzy]

So tried it and no-go.

From the Lahey manual could I not use blocksize as in: blocksize is a scalar default INTEGER expression. It must evaluate to the size, in bytes, of
the input/output buffer. And since an INTEGER defaults to 4 bytes then that jives with what you said except that is for Lahey, not Intel.

However, Intel IFORT says: 

The BLOCKSIZE specifier indicates the physical I/O transfer size in bytes for the file. It takes the following form:

BLOCKSIZE = bks

bks	Is a scalar numeric expression. If necessary, the value is converted to integer data type before use.

If you specify a nonzero number for bks, it is rounded up to a multiple of 512 byte blocks. The maximum valid value of BLOCKSIZE is 2147467264.

If you do not specify BLOCKSIZE or you specify zero for bks, the default value of 128 KB (131,072 bytes) is assumed. However, if you compile with the assume buffered_stdout option, the default blocksize for stdout is 8 KB.

So, I can't see how I can possibly use a block size of 4-bytes with IFORT. It defaults to 128 KB and if you provide a number like 4 bytes it will round up to 512 bytes.

[andrew_4619]

byterecl is not block size it is the UNITS so 1 can be 1 byte or 4 bytes (1 32 bit word)

[sindizzy]

I may be out of my element here. So let me see If I'm on the right path:

In Lahey RECL=22000 this would mean records are 22,000 bytes long because the unit default is 1-byte.

To get the same behavior in Intel IFORT I would have to do RECL=5500 since the default unit is 4-bytes.

or with the byterecl switch I could have RECL=22000 since that makes the unit 1-byte?

The var BLOCKSIZE in this case is not of concern?

[andrew_4619]

Yours maths is correct.

[sindizzy]

Let me try this approach and I will report back.

[andrew_4619]

Note Steves comment, I will also add that binary files created from one compiler in general can be different with another compiler. It is not a fully portable format.

[Steve_Lionel]

While it is true that FORM='UNFORMATTED' files are not necessarily portable, use of ACCESS='DIRECT' reduces the typical differences. I do wonder if the file was written using DIRECT using the Lahey compiler.

Given that this program reads only the first record, the RECL units may not matter. At this point I'd want to see the data file, or perhaps a hex dump of the first 100 bytes or so.

[sindizzy]

I was looking for the source file that writes the data but have come up empty. All I have is the file the reads the data.

I only posted the first record read as that is where I am having difficulties. The rest of the algorithm depends on what I read from the first record. Once I have that resolved, reading the other records is straightforward. In fact, if I iterate through REC=2 upwards I can parse out the data. It's just REC=1 that's giving me fits.

[GVautier]

In your sample code, I dont think that the record length as any effect when reading only a real unless the file as a length smaller than 4*2200 bytes. In that case, the read statement will fail. Try to add iostat= to the read statement and check the result.

Are you sure that the default real size is the same for the Lahey compiler and the intel compiler? An hexdump of the first bytes of the file may give a clue.

[sindizzy]

I only posted the first record read as that is where I am having difficulties. The rest of the algorithm depends on what I read from the first record. Once I have that resolved, reading the other records is straightforward. In fact, if I iterate through REC=2 upwards I can parse out the data. It's just REC=1 that's giving me fits.

[sindizzy]

Ok folks I appreciate all the help. The REAL        RCNT defaults to a 4-byte (32-bit) float in Lahey.

I use pretty standard Lahey switches to compile

lf90 c:\src\NEWCNT.for -win -FULLWARN -chk -w > c:\src\out.txt

In my test code I create a file just as before and then write out a single record. The 13419. is the number that I get when I read the existing BIN file so just used that to pipe into my TST.BIN file.

    IMPLICIT NONE    
    CHARACTER * 255  TstFile
    REAL  RCNT
    TstFile='C:\docs\TST.BIN'
    OPEN (UNIT=8, FILE=TstFile, ACCESS='DIRECT', RECL=22000,
   +      STATUS='REPLACE', FORM='UNFORMATTED')
    RCNT=13419.
    WRITE(8,REC=1)RCNT
    CLOSE(8)

The VS hex editor shows this with the existing binary file on the left and my TST.BIN on the right. They look the same. So, if every record is 22000 bytes and we read the first 4 bytes shouldn't that be a 13419. ?

[sindizzy]

In Intel IFORT without byterecl I get the following if I use RECL=20000 or RECL=5500
Same if I do use byterecl

[GVautier]

In the original file, the 4 first bytes neither represent an IEEE 4 bytes number nor 13419 in a 4 bytes integer. The tst.bin generated by the Lahey compiler as I understand and the original file are identical. So it could mean that the representation of floating point numbers, at least stored in direct files, are not compatible with IEEE representation. Have you tried to post a question on a Lahey Fortan forum?

https://baseconvert.com/ieee-754-floating-point

In Intel Fortran, the recl value in direct files doesn't affect the structure of the file. It is just a value allowing to access the file by records of the declared length to read /write in the file by whole record. So you can open this file with any record length and read data in the first record. The correct record length is only mandatory to access other records.

[sindizzy]

Lahey was sold to Fujitsu and is now defunct. Thus my drive to use a newer compiler.

[mecej4O]

Lahey sold different versions of Fortran compilers, some of their own creation, and some that were Lahey ports of Fujitsu Fortran compilers to Win32 and Linux-x64. As far as I know, Fujitsu never owned any part of Lahey Computers.

[GVautier]

I just found a clue.

The second and third bytes 55F0 is the decimal representation of the number 22000. So it is the record length and it is stored by Lahey Fortan in each record or in only the first. Can you change you test program with a record of 128 for example and post the hex dump of the first 256 bytes of tst.bin file.

https://community.intel.com/t5/Intel-Fortran-Compiler/Reading-Unformatted-Files-Compiled-with-Lahey-Fortran/m-p/810374 

[sindizzy]

Very interesting. I think I was lured in by assuming all Fortran code would produce the same output. I think it depends on how the particular software company implemented certain things.

New code:

    IMPLICIT NONE    
    CHARACTER * 255  TstFile
    REAL  RCNT
    TstFile='C:\docs\TST128.BIN'
    OPEN (UNIT=8, FILE=TstFile, ACCESS='DIRECT', RECL=128,
   +      STATUS='REPLACE', FORM='UNFORMATTED')
    RCNT=13419.
    WRITE(8,REC=1)RCNT
    RCNT=200.
    WRITE(8,REC=2)RCNT
    CLOSE(8)

And here is the hex dump. Would F7 be a possible header marker? 80=128

13419.0 = 0x00ac5146 (big endian)

200.0 = 0x00004843

[GVautier]

It becomes clearer. The first record looks like a header with a 1 byte marker and the record length in the 4 to 8 following bytes.

In order to read this file with a program compiled with Intel Fortran, you must shift your record number to skip the header : REC=2 for the first record and so on.

[sindizzy]

Very interesting indeed. 

Lahey documentation says "recl is a scalar default INTEGER expression. It must evaluate to the record length for a file
connected for direct access, or the maximum record length for a file connected for sequential
access."

and "INTEGER 4* Range: -2,147,483,647 to 2,147,483,647" is the default so 4-bytes = 32-bit integer.

But I realize that its "backward" meaning I read that as 00 00 00 80. IS that what is meant as big-endian?