Hello,
Im
developing a distributed application in Fortran using MPI. I would like to
transfer a file by network from a process to the others. The file is an
unformatted file created by another Fortran application. I have a complex
subroutine which can parse this file and fill plenty of structures and
variables.
The size of
these variables is about 10 times bigger than the file itself. So in order to
minimize the network load, Im trying to transfer the raw file and parse it in
each process.
My first
solution was to read the file in binary mode and fill a buffer (basically an
integer array) and broadcast this buffer. It works fine and any of the process
can write back on the disk the exact same file for testing purposes.
But many processes
are running on each machine so for performance and concurrency problems I search
to avoid this hard drive writing operation and make my parsing routine use
directly the buffer in memory. I tried using a memory pipe and writing in it in
binary mode and opening this pipe in unformatted mode and passing the
corresponding unite as it to my parsing routing, but the opening fails.
The other
solution would be to directly read the file in unformatted mode without parsing
it, but I have no idea what kind of data I could use to store the records.
Any help
about my problems or another way to achieve rapidly this data transfer would be
greatly appreciated.
Nicolas
