Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
41 Views

Difference between COMMITQQ, PXFFFLUSH, and FSYNC

In rare cases when my code crashes soon after formatted IO, some data are not actually written to disk. I am currently calling `FLUSH` on all files (before the crash) but it is apparently insufficient. The disk is an HPC lustre storage system. My current understanding is that there are two possible problems:

  1. The flush operation hasn't fully completed. From here (https://software.intel.com/en-us/node/679281), flush is nonblocking.
  2. The flush completed but the file isn't actually physically on disk yet. From here (https://gcc.gnu.org/onlinedocs/gfortran/FLUSH.html), flush only affects the runtime IO buffer, not the actual disk.

Ideally, I would like a portable way to perform a blocking flush directly to the disk. I mostly use intel, though, and it seems there are many compiler-specific routines. I have found 3 possible solutions:

  1. Call the PXFFFLUSH routine (https://software.intel.com/en-us/node/679697#42E1D0BC-BEA7-400D-A0A5-303ADA415EAA), which is an intel-specific flush directly to disk. Is this nonblocking like the regular flush?
  2. Call the COMMITQQ routine (https://software.intel.com/en-us/node/679139#DCC1CED7-520A-462C-B48B-8ED5B29C5EC4), which is an intel-specific blocking flush. It also seems to force a direct write to disk
  3. Grab the POSIX file descriptor and call FSYNC, following this example (https://gcc.gnu.org/onlinedocs/gfortran/FLUSH.html). Grabbing the POSIX descriptor appears to be compiler-specific, but the FSYNC call itself should be portable.

My question is: what are the differences between PXFFFLUSH, COMMITQQ, and FSYNC? Are they all blocking? Which (if any) would be appropriate for my issue?

0 Kudos
0 Replies