Asynchronous Write

[dannycat]

I wish the save sections of data within my application at regular intervals so that an "Undo" type operation is possible. Currently this works fine but the user has to wait while the data is written to the undo file. I thought that the asynchronous write option would be ideal for the scenario so I've coded all open and write statements to include the ASYNCHRONOUS='YES' option. When the program tries to save the file it returns an error status = 8 part way through a loop that's writing a section of the data. The error code implies that there is a error of the program or a bug that should be reported.

Am I using the feature correctly, and if so has anyone got any ideas what may be going wrong.

[Steven_L_Intel1]

Are you doing a WAIT before modifying any of the data being written? How exactly are you coming up with an error status of 8? I don't think that's a normal Fortran I/O status.

[dannycat]

Quoting - Steve Lionel (Intel)

Are you doing a WAIT before modifying any of the data being written? How exactly are you coming up with an error status of 8? I don't think that's a normal Fortran I/O status.

Steve,

The data is not modified during the write process therefore I have not used the WAIT statement. The error status 8 is returned via the iostat of theWRITE statement. This is inside a do loop. The iostat returns 0 for the first 93 iterations (from a total of over 10000) of the loop and then returns the with the error. The data does not change at all. I guess that I should try the produced a simple example showing this behaviour. The documentation states that error status 8 implies that there is a bug of some sort (if the program coding is ok which I think it is!).

[Steven_L_Intel1]

An example would be helpful - thanks.

[jimdempseyatthecove]

Quoting - dannycat

Steve,

The data is not modified during the write process therefore I have not used the WAIT statement. The error status 8 is returned via the iostat of theWRITE statement. This is inside a do loop. The iostat returns 0 for the first 93 iterations (from a total of over 10000) of the loop and then returns the with the error. The data does not change at all. I guess that I should try the produced a simple example showing this behaviour. The documentation states that error status 8 implies that there is a bug of some sort (if the program coding is ok which I think it is!).

Would you by chance, either explicitly or implicitly, performing a delete or reallocation of any of the data while being written?

Note, if this is a mixed language application, and if the C++ part is using containers, and if the container gets expanded, you may experience an implicitreallocation.

Instead of Asynchronous Write, you might condisidered using a seperate thread?

Jim Dempsey

[dannycat]

Quoting - jimdempseyatthecove

Would you by chance, either explicitly or implicitly, performing a delete or reallocation of any of the data while being written?

Note, if this is a mixed language application, and if the C++ part is using containers, and if the container gets expanded, you may experience an implicitreallocation.

Instead of Asynchronous Write, you might condisidered using a seperate thread?

Jim Dempsey

Thanks for the suggestion, I'll maybe use this if I don't solve the asynchronous write problem. My application is of a standard Windows type and the write statements where the problem occurs is in a subroutine selected when the user selects the save file option. I was wondering if windows is causing something in the background to trip the asynchronous write. Does the WAIT statement wait until the data has been output to the virtual buffer before allowing the process to continue or does wait until the data has been written to disk? If it is the later I don't see an advange to using asynchonous WRITEs. I am looking into it further though and will try the produce a simple example to demonstrate the problem when I get time.

[jimdempseyatthecove]

Quoting - dannycat

Thanks for the suggestion, I'll maybe use this if I don't solve the asynchronous write problem. My application is of a standard Windows type and the write statements where the problem occurs is in a subroutine selected when the user selects the save file option. I was wondering if windows is causing something in the background to trip the asynchronous write. Does the WAIT statement wait until the data has been output to the virtual buffer before allowing the process to continue or does wait until the data has been written to disk? If it is the later I don't see an advange to using asynchonous WRITEs. I am looking into it further though and will try the produce a simple example to demonstrate the problem when I get time.

SOP is to place the WAIT(IOUNIT,...) in front of the (asynchronous) WRITE(IOUNIT,...)

This will wait for the prior pending write (if any) and also return potential error conditions from the prior pending write.
subroutine dump
...
wait(iounit, IOSTAT=IOerror)
if(IOerror) goto 999
write(iounit, ...
...

Jim Dempsey

[dannycat]

Quoting - jimdempseyatthecove

SOP is to place the WAIT(IOUNIT,...) in front of the (asynchronous) WRITE(IOUNIT,...)

This will wait for the prior pending write (if any) and also return potential error conditions from the prior pending write.
subroutine dump
...
wait(iounit, IOSTAT=IOerror)
if(IOerror) goto 999
write(iounit, ...
...

Jim Dempsey

Jim/Steve, I've added the WAITstatement to the loop and that results in a successful completion of the loop (65775 steps), although it is very slow (at least in debug mode where the output window displays that "Win32 thread (xxxx) has exited"). This seems to defeat the object of using this feature. At the moment it calls the WAIT every loop iteration step which is probably too frequent. What worries me is that if I call every 50 steps or (whatever number) I can't rely upon this working every time. Where is the data written in an asynchronous WRITE before being written to the file? i.e. does it fill up a buffer or use the stack which may affect the behavior?

Chapman's FORTRAN 95/2003 book indicates that writes do not require the WAIT statement but this does not seem to be the case, except before modifying any of the variables being written later in the program.

Incidently when I let the program proceed past the loop (that was initially failing) it crashes with an Access Violation at a later asynchronous WRITE statement in the _for_waitid low level routine (I assume it is trying to write a diagnostic messageusing _for_emit_diagnostic according to the call stack).

What effect does a normal (synchronous) WRITE have with a series of asynchronous WRITE's. (I'm pretty sure this is not the case in my example).

I am going off the idea of using this feature and look at using a separatethread instead, which at the moment is unknown territory for me.

[Steven_L_Intel1]

You may have found a bug in the implementation. Can you show a small but complete example that demonstrates the problem?

[jimdempseyatthecove]

This would indicate that you should use a seperate thread for writing. If your code is currently not using OpenMP you can QED the implementation

[cpp]program Foo
  use stuff
  use omp_lib
  call FooInit
  ! initializations to be done
  ! prior to entering parallel sections
  ComputeDone = .false.
  LastStepDone = 0
!$OMP PARALLEL SECTIONS NUM_THREADS(2)
  call DoCompute
!$OMP SECTION
  call DoIO
!$OMP END PARALLEL SECTIONS
  call FooFini
end program Foo

subroutine DoCompute
  use stuff
  ...
  do iStep=1,iLast
    call DoStep(iStep)
    LastStepDone = iStep
  end do
  ComputeDone = .true.
end subroutine DoCompute

subroutine DoIO
  use stuff
  integer :: NextDumpStep = 1
  call DoIoInit
  do while(.not.ComputeDone)
    if(LastStepDone .ge. NextDumpStep) then
      Call Dump(NextDumpStep)
      NextDumpStep = NextDumpStep + DumpInterval
    else
      call sleep(SleepInterval)
    end if
  end do
  call DoIoFini
end subroutine DoIo[/cpp]

Jim Dempsey

[dannycat]

Quoting - jimdempseyatthecove

This would indicate that you should use a seperate thread for writing. If your code is currently not using OpenMP you can QED the implementation

[cpp]program Foo
  use stuff
  use omp_lib
  call FooInit
  ! initializations to be done
  ! prior to entering parallel sections
  ComputeDone = .false.
  LastStepDone = 0
!$OMP PARALLEL SECTIONS NUM_THREADS(2)
  call DoCompute
!$OMP SECTION
  call DoIO
!$OMP END PARALLEL SECTIONS
  call FooFini
end program Foo

subroutine DoCompute
  use stuff
  ...
  do iStep=1,iLast
    call DoStep(iStep)
    LastStepDone = iStep
  end do
  ComputeDone = .true.
end subroutine DoCompute

subroutine DoIO
  use stuff
  integer :: NextDumpStep = 1
  call DoIoInit
  do while(.not.ComputeDone)
    if(LastStepDone .ge. NextDumpStep) then
      Call Dump(NextDumpStep)
      NextDumpStep = NextDumpStep + DumpInterval
    else
      call sleep(SleepInterval)
    end if
  end do
  call DoIoFini
end subroutine DoIo[/cpp]

Jim Dempsey

Thanks Jim I'll try this when I get the opportunity.

Steve,

I've written a simple example (attached) which demonstrates the problem. If you set the flags forWAIT to be enabled the code will complete but is very slow. Running as is it fails at around the 1000th iteration. Taking the IOSTAT=ier out of the write statements will produce the Access violation errors at the same place.

[dannycat]

Quoting - Steve Lionel (Intel)

You may have found a bug in the implementation. Can you show a small but complete example that demonstrates the problem?

Steve, I've attached a simple demonstration codeexample showing the problem with Asynchronous WRITE I have described above.

[jimdempseyatthecove]

Quoting - dannycat

Thanks Jim I'll try this when I get the opportunity.

Steve,

I've written a simple example (attached) which demonstrates the problem. If you set the flags forWAIT to be enabled the code will complete but is very slow. Running as is it fails at around the 1000th iteration. Taking the IOSTAT=ier out of the write statements will produce the Access violation errors at the same place.

Dannycat,

In looking at your sample program as setup you have

[cpp]lasyn = .true.    ! Enable Asynchronous Write
!lasyn = .false.    ! Disable Asynchronous Write
!setwait = .true.  ! Enable wait
setwait = .false. ! Disable Wait
[/cpp]

Which causes you to perform asynchronous writes with no waits. For asychronous writes the completion condition for each write is buffered (as opposed to the condition code of the last write). Apparently this buffer is overflowing. Periodically you should flush the pending buffered wait conditions.

Jim Dempsey

[dannycat]

Quoting - jimdempseyatthecove

Dannycat,

In looking at your sample program as setup you have

[cpp]lasyn = .true.    ! Enable Asynchronous Write
!lasyn = .false.    ! Disable Asynchronous Write
!setwait = .true.  ! Enable wait
setwait = .false. ! Disable Wait
[/cpp]

Which causes you to perform asynchronous writes with no waits. For asychronous writes the completion condition for each write is buffered (as opposed to the condition code of the last write). Apparently this buffer is overflowing. Periodically you should flush the pending buffered wait conditions.

Jim Dempsey

Jim,

The difficulty is specifying the frequency of the waits. If the buffer is full the implementation should flush it automatically otherwise you have to somehow work out the buffer size and estimate the amount of data written. This may have portability issues. If the waits are too frequent the application is slow and will probably take longer than a normal write.