Delayed buffered file io

[Petros]

Hi everybody,

I have a time-domain simulator written in Fortran 2003. I have some observables that are written into a file with the purpose of post-prossesing. The software is 2-4 times faster when not saving these outputs.

I want to delay as much as possible the writting of the values to the output file (if possible after the simulation is over). The reason is that during the simulation, there is an automatic procedure which decides if the simulation is stable (thus, no post-processing needed) and exits immediatly (with a STOP). So if the case is stable, I would save a lot of time.

The outline of the code is:

[fortran]open(newunit=output_file,file=filenm,form='unformatted',buffered='yes',status='replace',buffercount=127,BLOCKSIZE=4194304)

do while (.not. end_simul)

  t=t+h

  call solve(t,end_simul)

  call check_early_stop(iExit)

  if(iExit)STOP 'Early exit'

  do i=1,number_of_outputs

    if(selected(i))write(output_file)output_value(i)

  enddo

enddo

close(unit=output_file)[/fortran]

RAM memory is not an issue since the program uses less than 1GB of ram while the computer has more than 64GB. I run on a Interlagos Opteron 32-cores. I used the buffercount and blocksize to try and increase the buffer so that it fits all the file and the writting is delayed. I saw very little difference.

All outputs are double precission.

I use "Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 13.1.0.146 Build 20130121". I'm running Debian Wheezy (6).

Any help on how to speed up the writting or delay it is welcome!

Thanks in advance,

Petros

[Steven_L_Intel1]

Suggestion 1 - use asynchronous I/O.  This works best if the writes are not small (as in single values).

Suggestion 2 - save the values in an allocatable array and write them when done. This is really what you want, with the main issue being that you (probably) can't predict how many values you'll need. If you can estimate this and "overprovision", and add the ability to resize the array if needed (requires allocating new one, copying old values, then using MOVE_ALLOC to reassign the allocation), that should work best.

Suggestion 3 - try not to do such small writes!

[Petros]

Hi Steve!

Thanks for the straightforward and complete answer. I know that writing such small values is bad, but I inherited this part of the code and I'm trying to make it better.

I will declare a double precission vector to put all the output values of all the simulation and make write in the end. I can estimate the size of the vector as num_outputs*(time horizon)/time-step. With a margin of 10% upwards (because some times there is step reduction). Then I should use write(output_file)big_vector(1:final_used_element)?

I'll get back to you with the results,

Petros

[Steven_L_Intel1]

Yes, that's what I would recommend.

[Petros]

Steve Lionel (Intel) wrote:

Yes, that's what I would recommend.

Hi Steve,

Well, I tried the suggestion and it's indeed a lot faster. I have a double precission vector buffer which I allocate to the size needed and then I have save sequentially everything I would write to the file. In the end I write all together. I have a problem with the post-processing though. In the post processing I have a loop:

[fortran]do i=1, numOfOutputsPerTimeStep

read(out_file)buffer(i)

enddo[/fortran]

If I write the output file as:

[fortran]write(out_file)outBuffer(1:totalNumberOfOutputs)

close(out_file)[/fortran]

then the post-processing loop above crashes with a message: "forrtl: severe (24): end-of-file during read, unit 10, file /home/p3tris/out_file.trj"

If I write the file as:

[fortran]do i=1, totalNumberOfOutputs

write(out_file)outBuffer(i)

enddo

close(out_file)[/fortran]

then it works perfectly.

In the unformatted file there is some notion of newline? Could I modify the reading part to not crash with the first way of writting?

Thanks again,

Petros

[Steven_L_Intel1]

read (out_file) buffer(1:numOfOutputsPerTimeStep)

and do away with the loop.

You're using unformatted I/O (good), so there is no newline. However, there is a record length. As an alternative, open the file with ACCESS='STREAM' and you can keep the code the way it is.

[Petros]

Steve Lionel (Intel) wrote:

read (out_file) buffer(1:numOfOutputsPerTimeStep)

and do away with the loop.

You're using unformatted I/O (good), so there is no newline. However, there is a record length. As an alternative, open the file with ACCESS='STREAM' and you can keep the code the way it is.

Hi, sorry for the delayed answer. Well, I tried with stream but a problem occurred. At the beginning of the output file I write the names of the output variables (string). That is I have:

[fortran] do i=1,number_of_outputs

if(selected(i))write(output_file)output_value_name(i)

enddo[/fortran]

And then all the following procedure of writing in the simulation loop.

The same when reading back, I first have to read the names of the variables and then their values.

When I use ACCESS='STREAM' both for writing and reading in, it blocks at the first read (string) and stays there. No error no continuing.

Petros

[Steven_L_Intel1]

Please show me a small but complete sample program that demonstrates the problem.  But I'll comment that ACCESS='STREAM' uses the size of the variable in the I/O list.

[Petros]

Steve Lionel (Intel) wrote:

Please show me a small but complete sample program that demonstrates the problem.  But I'll comment that ACCESS='STREAM' uses the size of the variable in the I/O list.

Hi Steve,

I managed to make things work. As always a human factor mistake :) I attach 2 files, one writes the file and one reads it. Inside each file there you can use First option -> writting/reading with a loop and Second option -> writting/reading all together and using stream.

It seems to work OK, but just in case if you see something strange let me know!

Thanks for all the trouble,

Petros

PS: For some reason uploading the f90 files was failing, I've put them here --> https://dl.dropboxusercontent.com/u/258337/intel%20forum/write.f90 and https://dl.dropboxusercontent.com/u/258337/intel%20forum/read.f90 

[Petros]

Steve Lionel (Intel) wrote:

Please show me a small but complete sample program that demonstrates the problem.  But I'll comment that ACCESS='STREAM' uses the size of the variable in the I/O list.

Steve Lionel (Intel) wrote:

Please show me a small but complete sample program that demonstrates the problem.  But I'll comment that ACCESS='STREAM' uses the size of the variable in the I/O list.

Hi Steve,

I managed to make things work. As always a human factor mistake :) I attach 2 files, one writes the file and one reads it. Inside each file there you can use First option -> writting/reading with a loop and Second option -> writting/reading all together and using stream.

It seems to work OK, but just in case if you see something strange let me know!

Thanks for all the trouble,

Petros

PS: For some reason uploading the f90 files was failing, I've put them here --> https://dl.dropboxusercontent.com/u/258337/intel%20forum/write.f90 and https://dl.dropboxusercontent.com/u/258337/intel%20forum/read.f90 

[Steven_L_Intel1]

This looks fine, though you could also write and read the outDataNames array in a single operation as you do the data, rather than in a loop. I doubt it makes a signficant difference, though.