Convert REAL*8 Unformatted Sequential file

[daweiwan]

Hello, I am working on porting a Fortran program from a VAX to PC. The program needs to read data from an

Unformatted Sequential file as follows:

IMPLICIT REAL*8(A-H,O-Z)

DIMENSION R(4,80,6,46)

...

OPEN (UNIT=11, ERR=998, STATUS='OLD',

ACCESS='SEQUENTIAL', FORM='UNFORMATTED', IOSTAT=IERRA, FILE='water')

READ(UNIT=11, IOSTAT=IERRA) R
.........

According to this post's illustration, http://origin-software.intel.com/en-us/forums/showthread.php?t=45634

I am modify this program as follows:

IMPLICIT REAL*8(A-H,O-Z)

DIMENSION R(4,80,6,46)

...

IPRNT = 4

OPEN(UNIT=11,ERR=998,STATUS='OLD',ACCESS='SEQUENTIAL',

FORM='binary',convert='VAXD',FILE='water')

open(unit=iprnt,ERR=998, status='replace',IOSTAT=IERRA,file='wdise.TXT')!

READ(UNIT=11, IOSTAT=IERRA) R

.........

I meta problem when Idebug the program ,the prolem is:"forrtl: info(95) floating point conversion failed".

So confused I am, Would you please give me some advice to solve this problem ,Thanks.

David.wan

[Steven_L_Intel1]

Can you attach the data file, or at least a hex dump of the first 100 bytes or so? Did the file come from a VAX or an Alpha? It's possible that the REAL*8 data is G_float meaning you'd need 'VAXD'.

Does the program read any of the data correctly?

[daweiwan]

Ok,Maybe I need to find another way to solve this problems

[Steven_L_Intel1]

How did you transfer the data file from the VAX system? It looks to me as if the record lengths have been lost.

Here's an article I wrote on this many years ago, and I've attached the programs it mentions. Perhaps this will be helpful to you.

Description: Unlike FORMATTED files, Fortran SEQUENTIAL UNFORMATTED files vary in data layout among various compiler vendors and platforms. This article summarizes how to read UNFORMATTED files from various combinations of compilers and platforms.

The Visual Fortran SEQUENTIAL UNFORMATTED file layout is one that is very common on UNIX systems, including DIGITAL UNIX. Each Fortran "record" is preceded and followed by a 32-bit integer containing the record length (the integer is stored in the "little-endian" layout, with the least-significant bit in the lowest addressed byte.) This record length is required to apply Fortran record semantics, including the ability to use BACKSPACE. Unfortunately, this is not correctly documented in the Programmer's Guide - the layout described there is actually that of Microsoft Fortran PowerStation. This will be corrected in future editions.

DIRECT UNFORMATTED files do not typically have embedded record length information and can usually be read/written without special actions.

Microsoft Fortran (including Fortran PowerStation and Fortran 5.x for DOS)

These files can be read (and written) directly by enabling the /fpscomp:ioformat switch (Compatibility..PowerStation..I/O Format setting). Note that this is the default if you use the FL32 command from a command prompt.

DIGITAL Fortran for DIGITAL UNIX

These files can be read and written directly with no special actions.

Other UNIX systems

For HP, IBM, Sun, SGI and other big-endian UNIX systems, open the file with the CONVERT='BIG_ENDIAN' option. For little-endian UNIX systems (such as Linux on X86 processors), no special action is typically needed.

DIGITAL Fortran for OpenVMS

The default SEQUENTIAL UNFORMATTED file layout on OpenVMS is called 'SEGMENTED'. A logical record may be split across multiple physical records - each has a 16-bit prefix of flags indicating whether that physical record is the first, last middle or only in the logical record. While Visual Fortran CAN read these, if the file is opened with RECORDTYPE='SEGMENTED', in practice this is difficult because you must transfer the data in such a way as to preserve the file system's record lengths, and typical methods such as FTP remove this information.

One way is to use a program on the OpenVMS system to convert the file to a format which can be easily transferred and read by Visual Fortran. To convert an OpenVMS 'SEGMENTED' file to a form which can be read by Visual Fortran, use segmented_to_unix.for (note that the Visual Fortran layout is the same as is used by UNIX systems). To convert a Visual Fortran file so that it may be read by Fortran on OpenVMS, use unix_to_segmented.for.

Another way is to prepare a SEGMENTED file for transfer by changing the file's recordtype to be fixed-length, 512 byte records, using the command:

$ SET FILE /ATTRIBUTES=(RFM=FIX,LRL=512) file.dat

This file can then be copied to the Windows system using normal FTP in binary mode. On the Windows system, open the file using RECORDTYPE='SEGMENTED'. This method cannot be used for files that were opened with RECORDTYPE='VARIABLE' explicitly specified.

If the file contains floating data and was written on a VAX or on an Alpha system with the default VAX floating types, you will need to specify CONVERT='VAXD' or 'VAXG' (as appropriate) when opening the file using Visual Fortran. This will automatically convert the data in scalars and arrays (though not in RECORD structures or derived types.)

[Steven_L_Intel1]

If all the records are the same length, you can recover by opening the file with RECL=176640, RECORDTYPE='FIXED'.

[mecej4]

I noticed something curious about the binary data file.

To hold the contents of the array R, with the declared dimensions of (4,80,6,46) would require (4 X 80 X 6 X 46)numbers X 8 bytes/number = 706560 bytes, plus any record length and header bytes. The data file "water" that you attached contains 584252 bytes, which amounts to not even 7 bytes per double precision real.

There is something inconsistent here.

Secondly, until one gets things to work properly, in cases like this it would be better not to include ERR= and IOSTAT= options in READ statements -- let the system tell you the errors it runs into while trying to read the file.

[daweiwan]

Steve,Thanks for your help. I compiletheprogram (segmented_to_unix.for) using Compaq Visual Fortran 6.60 on my PC ,but it occuringa lot of problem.Therefore,Ihave a question. Whether the program must be run in the VAX computer ? not in PC, and how to use the command in CVF, '$ SET FILE /ATTRIBUTES=(RFM=FIX,LRL=512) file.dat', thanks.
Anyway,we can't find the vax computer easily, so other solutions could befound ?

[Steven_L_Intel1]

The segmented_to_unix program, as well as the SET FILE command, must be run on the VAX. If you no longer have access to the VAX, consider my suggestion of using RECORDTYPE='FIXED', but also look at the issue of the file size.

[daweiwan]

Oh,bad luck, and I am using RECORDTYPE='FIXED' to convertthe file ,however, itcan'tberead.

So, how I can solve this problem? Does some onein the same situation ? Thanks all the same ,steve.

[anthonyrichards]

Addressing mecej4's objection regarding the inconsistency between the length of the binary data in the file that you attached and the number of bytes required to store the number of REALs each 8 bytes long in the array R - Clearly you have attached a truncated file.

The array you are reading is a REAL*8 dimensioned (4,80,6,46). Taking the first two dimensions and multiplying, you get 4*80*8=2560 bytes. dividing by 16 gives 160 decimal, or A0 hexadecimal. If you open the file as binary (i.e. not auto) in the VS editor, you will soon see that the following repeats every 160 lines (or A0 hex), assuming 16 bytes per line displayed:

00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 80 40 00 00 00 00 00 00 00 00 00 00 00 00
00 00 80 40 00 00 00 00 00 00 00 00 00 00 00 00
00 00 7F 40

Using this flag, you can at least isolate groups of 320 real values. Clearly there should be 6*46 = 276 such groups. This indicates that the array was almost certainly written in column major order starting with the first index and writing the data as sets of 2-dimensional arrays.

Within each group, the first index should vary fastest, so there should be natural groups of 4*8 bytes (possibly with spacer bytes).

[daweiwan]

Hi,anthonyrichards.Thanks for yourenthusiastic help. And I learn it carefully,But my knowledge for CVF is limited.

Especially, "Within each group, the first index should vary fastest",how can I realize this function?

So, Could you pleaseprovideme some specificsolutions?And I can obtain this data accurately . Thank you.

Best wishes to you.

[mecej4]

Allow me to suggest an altogether different direction. Given that the file from the VAX is truncated or mutilated, of uncertain format/type, and my have lost some content during transfer by FTP ftom an RMS file system to a byte-stream type file system (e.g., Unix, Windows), it is justifiable to ask: "What is supposed to be in the file 'water'"?

If, as the file name suggests. it contains some properties of water, it should be easy to recreate it ab initio . Software for calculating these properties is widely available.

[anthonyrichards · Accepted solution]

Your original attachment is clearly truncated and covers only part of the REAl*8 (4,80,6,46) array. However, examination of the file in the VS editor shows a regular 2044 byte block structure, with each block having two bytes padding as the first two bytes (apart from the first, which has 4 bytes of padding). Within this regular block structure can be perceived a 2560-byte repeat pattern, corresponding to 4*80*8 bytes, which matches what you would expect if the file contains data for an array of REAL8* arrays each of dimension (4,80).

I attach a small program which I have written to process the 2044 blocks to create a new binary file with 2042-byte records. This new binary file is then read to recreate the REAL*8 array. The program does no conversion of the data. It assumes that the ASCII code used in the file matches that used by windows and that the byte data are arranged MSB first, LSB last, i.e. if a number starts at byte N, byte N is assumed to be the MSB and byte N+7 is assumed to be the LSB. In a data record, byte N is to the left of byte N+1 and subsequent bytes.

The program should be run in debug mode so that you can examine the content of the Real array R when the program reaches the PAUSE, to see if they make sense to you (assuming you have some idea of what magnitudes the numbers should be). Note that there is only enough data in the file to create an array of values of dimension (4,80,6,37).

I attach the program for you to do with as you wish.

[daweiwan]

Thanks so much to Anthonyrichards and Steve, and others. I got much help from yours, Friends. Now, I find the

solution for this problem.