The best/efficient way of writing a huge 3D array into a binary file

[dyun]

Hi Everyone,

I am working on saving and reading a huge 3D array to and from a binary file. Using intel fortran compiler, the size of the array varies from 128x128x128 to 1024x1024x1024.

As far as I know, this array can be saved:

1.) elements by elements;

2.) row by row (or column by column, if see this 3D array is composed by a sequence of 2D arraies), or

3.) be saved as a single record.

But I am confused with which one of those three methods is the best, or in other words the most efficient, considering the consuming time (spend on saving and reading), accuracy and size of stored file (if different way will bring differences).

Thanks for your answers, comments and discussions.

[Steve_Lionel · Accepted solution]

For file size and "accuracy", a file opened with FORM="UNFORMATTED" is the only reasonable choice.

I would suggest that writing the whole array would be the most efficient in terms of time and resources. You may want to play with the BLOCKSIZE= value in OPEN to see if larger or smaller values make a difference. The default is 128KB and larger writes might end up more efficient. I assume you aren't interested in interweaving I/O and application work.

[dyun]

Thanks for Steve's answer. I agree with you. I did a simple test, saving and reading data in three ways and then output their cpu time respectively.