I have built a program which has to print large output files. When I print the output in a formatted ASCII file, the printing is almost instantaneous, taking a couple of seconds. However, when I run the program and try to print the SAME EXACT output to a binary file, the printing is extremely slow, taking more than 5 minutes
I wanted to ask if there is an explanation for such discrepancy, and whether there is something I could do to speed up writing to a binary file.
Many thanks in advance.
Probably a mistake you made somewhere. An example program showing the issue would be helpful.
One thing I often saw was where the program OPENed the file for unformatted I/O and specified a RECL in bytes, with the user not knowing that the Intel (and DEC-heritage) default was that this value was in 4-byte units. This would cause four times as much data to be written, which was slower. This can be adjusted with the /assume:byterecl option.
I cannot find anything wrong with my code. Also, the RECL SHOULD be 8 bytes. The printed output is to be read by another app (for which I do not have the source), and this other app uses a RECL of 8 bytes.
Here is a small sample from my code:
do i1 = 1,numnp1
filposit = filposit + 1
end do !i1
The value of the integer iWORDsiz1 is equal to 8, and fileposit is an integer parameter that gives the position of each variable to be printed.
I also wanted to provide the following additional information:
The problem could be hardware-related(?). When I run the program in a different computer, the speed is reasonable: this means that the print to the binary file is not slow at all (it is only very slightly slower than the corresponding ASCII print, which is expected in my opinion). On the other hand, when I run the program in the PC for which I reported the issue, the binary print is extremely slow. By "extremely slow", I mean that, instead of taking 3-4 seconds, it takes about 10 minutes!!!
More interestingly, when run the program IN THE SAME COMPUTER (where binary print was very slow) but in a folder located in a DIFFERENT disk drive, the binary print is fast! Could this be attributed to the hardware somehow? I cannot imagine that there is a hardware malfunction in the disk drive, because I cannot see how a malfunctioning drive would only have slow write in binary files (and fast write in ASCII files). Could the slow speed have to do with, e.g., the way in which the hard disk is formatted?.
Perhaps I'm missing something, but "binary" reads and writes or ordinarily MUCH faster than ASCII reads/writes, depending on the details of the data content somewhat but nearly always, binary is faster. To read/write ASCII requires an added layer of conversion to/from internal representation that is not need typically for binary IO.
When you incorporate pos=, and depending on sharing mode, if shared it implies
Read (buffer), Modify (buffer), Write (buffer)
If on open you specified a large buffer size, this can take considerable time.
If (when) the file is opened for exclusive use (no sharing), the Read and Write occur only when necessary.
Also the flushing can be exacerbated when the file is located on a network share.
Can you post your open statement?
Can you tell us the difference between file destinations?
Can you tell us if Antivirus scans one destination folder and not the other?
For good measure, I would also try BUFFERED='YES' in your OPEN() statement (of the binary output file). There have been some recent changes in the buffering (including a possible performance regression) so that could affect your results.