Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
6 Views

The best/efficient way of writing a huge 3D array into a binary file

Jump to solution

Hi Everyone,

I am working on saving and reading a huge 3D array to and from a binary file. Using intel fortran compiler, the size of the array varies from 128x128x128 to 1024x1024x1024.

As far as I know, this array can be saved:

1.) elements by elements;

2.) row by row (or column by column, if see this 3D array is composed by a sequence of 2D arraies), or

3.) be saved as a single record.

But I am confused with which one of those three methods is the best, or in other words the most efficient, considering the consuming time (spend on saving and reading), accuracy and size of stored file (if different way will bring differences).

Thanks for your answers, comments and discussions.

0 Kudos

Accepted Solutions
Highlighted
Black Belt
6 Views

For file size and "accuracy",

Jump to solution

For file size and "accuracy", a file opened with FORM="UNFORMATTED" is the only reasonable choice.

I would suggest that writing the whole array would be the most efficient in terms of time and resources. You may want to play with the BLOCKSIZE= value in OPEN to see if larger or smaller values make a difference. The default is 128KB and larger writes might end up more efficient. I assume you aren't interested in interweaving I/O and application work.

Steve (aka "Doctor Fortran") - https://stevelionel.com/drfortran

View solution in original post

0 Kudos
2 Replies
Highlighted
Black Belt
7 Views

For file size and "accuracy",

Jump to solution

For file size and "accuracy", a file opened with FORM="UNFORMATTED" is the only reasonable choice.

I would suggest that writing the whole array would be the most efficient in terms of time and resources. You may want to play with the BLOCKSIZE= value in OPEN to see if larger or smaller values make a difference. The default is 128KB and larger writes might end up more efficient. I assume you aren't interested in interweaving I/O and application work.

Steve (aka "Doctor Fortran") - https://stevelionel.com/drfortran

View solution in original post

0 Kudos
Highlighted
Beginner
6 Views

Thanks for Steve's answer. I

Jump to solution

Thanks for Steve's answer. I agree with you. I did a simple test, saving and reading data in three ways and then output their cpu time respectively.

0 Kudos