stream files

[pnaecker]

We want to move files from OpenVMS to Linux, the data to be used with Intel Fortran on Linux. The data is being written in FORTRAN on the OpenVMS side.

We thought we would simply use

open (lun, form='unformatted', status='new', 'recordtype='stream', carriagecontrol='none')

when we create the file, and that we would then end up with a file that is just a stream of bytes, which would of course move to Linux without difficulty.

However, when we create this file, OpenVMS (or OpenVMS FORTRAN) seems to want to create the file with CR carriage control, so we get extra bytes in the file.

Does anyone have a recommendation on how to write data (big, ugly, blobs of data - no structure) using FORTRAN in a way that is portable across OSes? Is there a way to coax FORTRAN into not doing CR carriage control? Is that what the form='binary' recently added to Intel fortran is all about?

Thanks in advance.

Phil

[Steven_L_Intel1]

If it is a text file, you can use FTP PUT in ASCII mode to get the right thing done. If you're transferring the file some other way, then use RECORDTYPE='STREAM_LF' on VMS to write something Linux systems will consider a normal text file.

For binary files, it gets more complicated. Here is some text I wrote many years ago on the topic, and I've attached the program in question. While it is written in the context of Compaq Visual Fortran, the same applies to Linux. And yes, FORM='BINARY' just writes bits.

---

The default SEQUENTIAL UNFORMATTED file layout on OpenVMS is called 'SEGMENTED'. A logical record may be split across multiple physical records - each has a 16-bit prefix of flags indicating whether that physical record is the first, last middle or only in the logical record. While Visual Fortran CAN read these, if the file is opened with RECORDTYPE='SEGMENTED', in practice this is difficult because you must transfer the data in such a way as to preserve the file system's record lengths, and typical methods such as FTP remove this information.

One way is to use a program on the OpenVMS system to convert the file to a format which can be easily transferred and read by Visual Fortran. To convert an OpenVMS 'SEGMENTED' file to a form which can be read by Visual Fortran, use segmented_to_unix.for (attached - note that the Visual Fortran layout is the same as is used by UNIX systems). Another way is to prepare a SEGMENTED file for transfer by changing the file's recordtype to be fixed-length, 512 byte records, using the command:

$ SET FILE /ATTRIBUTES=(RFM=FIX,LRL=512) file.dat

This file can then be copied to the Windows system using normal FTP in binary mode. On the Windows system, open the file using RECORDTYPE='SEGMENTED'. This method cannot be used for files that were opened with RECORDTYPE='VARIABLE' explicitly specified.

If the file contains floating data and was written on a VAX or on an Alpha system with the default VAX floating types, you will need to specify CONVERT='VAXD' or 'VAXG' (as appropriate) when opening the file using Visual Fortran. This will automatically convert the data in scalars and arrays (though not in RECORD structures or derived types.)

[pnaecker]

Mucho gracias. That code looks simple enough.

Independently, someone else suggested that all we really needed to do was write and read the records as 'SEGMENTED' and $ SET FILE/ATTR=RFM:STM on the file after it's been written and before it is transferred. We tried this and it seems to work. I'm a little confused as to _why_ it works, however; the fact that it works seems to indicate that FORTRAN is writing the bits correctly, and if we do the SET FILE on VMS that simply prevents VMS from adding some sort of record semantics to the file as it passes the bits to FTP. We'll play a bit more. And your solution clarifies the meaning of those first few bits in each record of a segmented file - we couldn't quite figure that out.

Thanks.

[Steven_L_Intel1]

Yes, the suggestion you were given is similar to the one I showed setting the RFM to FIX. It accomplishes the same thing in the end - preventing VMS from stripping the record lengths.

The meaning of the bits should be in the compiler documentation.