Using MPI-I/O to Read the Double-Precision Entity from a Fortran Unformatted File

[Torgny_F_]

Hello!

I am trying to read a CFD mesh file through MPI-I/O. The file is a Fortran unformatted format with big-endianness, and it contains mixed variables of integer and real*8 (the file starts with block-size integers, followed by x,y,z coordinates of that block). I can manage to read first integers but the real entities are completely wrong or not so accurate. So I simplified the code to reproduce the same error. It writes one real value to the file in Fortran unformatted format and try to read it back in serial and in parallel (through MPI-I/O). Following (italic text) is the test code :

program readtest
implicit none
include 'mpif.h'

integer :: myrank,nproc,ierr,istatus(MPI_STATUS_SIZE),mpifile,rdsize
integer(kind=MPI_OFFSET_KIND) :: disp
character(len=80) :: mpifiname
double precision :: in,vals,valp

! Define MPI basics
call MPI_INIT(ierr)
call MPI_COMM_RANK(MPI_COMM_WORLD,myrank,ierr)
call MPI_COMM_SIZE(MPI_COMM_WORLD, nproc,ierr)

! Initialize
in = 1.0/7.0
vals = 0.0
valp = 0.0

! Write a serial files
open(10,file='Serial.dat',form='unformatted')
write(10) in
close(10)

! Serial file read
open(10,file='Serial.dat',form='unformatted',status='old')
read(10) vals
close(10)

! Read by MPI-I/O
mpifiname = 'Serial.dat'

disp = 0
call MPI_FILE_OPEN(MPI_COMM_WORLD, mpifiname, &
MPI_MODE_RDONLY, MPI_INFO_NULL, mpifile, ierr)
call MPI_FILE_SET_VIEW(mpifile,disp,MPI_BYTE,MPI_BYTE,"external32",&
MPI_INFO_NULL,ierr)
rdsize = 0
if(myrank == 0) rdsize = 1
call MPI_FILE_READ_ORDERED(mpifile, valp, rdsize, MPI_DOUBLE_PRECISION, &
istatus, ierr)
call MPI_FILE_CLOSE(mpifile, ierr)

write(*,*) 'Input: ',in,'Serial:',vals,' Parallel:',valp

call MPI_FINALIZE(ierr)

stop
end

If you compile with the big-endian option (I add '-convert big_endian' option for Intel compiler), the result by Intel MPI slightly differs (It seems to be the byte-related problem):

mpirun -np 1 ./a.out

Input: 0.142857149243355 Serial: 0.142857149243355 Parallel: 0.142857074737549

If I abandon the big-endian mode (i.e., replace MPI_FILE_OPEN's data representation to "native" + set disp=4 to skip the first 4-byte record marker of Fortran unformatted format + no extra compilation flag), MPI-I/O reads exactly the same value. However, since the mesh file has been given in big-endian format, I have to keep using '-convert big_endian' option.

Can you let me know what the problem is?

Best,

Jeff

[TimP]

Not in my realm of expertise, but http://beige.ucs.indiana.edu/B673/node181.html points out the limitation of 'external32'.

[James_T_Intel]

Hi Jeff,

I've been doing a few experiments with your code.  I took the value read by MPI-IO, and wrote it back out.  I then read this natively in Fortran, and got the original value back.  I'm going to check with our developers to see if there is a way around this.

Sincerely,
James Tullos
Technical Consulting Engineer
Intel® Cluster Tools

[James_T_Intel]

Hi Jeff,

Our developers believe this is a bug and will be investigating.

James.

[James_T_Intel]

Jeff,

After investigation, this does not appear to be a bug.  Instead, it is simply the difference in how Fortran and C read and write binary data.  The developer's response:

If you comment the lines below and add this code to the program:

 character :: valb(16)
 integer :: i

...
! call MPI_FILE_READ_ORDERED(mpifile, valp, rdsize, MPI_DOUBLE_PRECISION, &
! istatus, ierr)
 call MPI_FILE_READ_ORDERED(mpifile, valb, 16, MPI_BYTE, &
 istatus, ierr)

...
!write(*,*) 'Input: ',in,' Serial:',vals,' Parallel:',valp
write(*,'(2(a,z16),a,80(z2,x))') 'Input: ',in,' Serial:',vals,' Parallel:',(valb(i),i=1,16)

you may examine the contents of the file:

$ !mpiif
mpiifort -convert big_endian test.f90 ; mpirun -n 1 ./a.out
Input: 3FC24924A0000000 Serial:3FC24924A0000000 Parallel: 0  0  0  8 3F C2 49 24 A0  0  0  0  0  0  0  8

You may observe extra 4 bytes in the beginning of the file, which breaks "normal" byte sequence expected by MPI functions.

If you want to really check external32 functionality, you can write code in C.