File opened with DISPOSE='DELETE' not deleted.

[jimdempseyatthecove]

I have a program that creates temporary files with DISPOSE='DELETE'

 Open(newunit=MergeInfo(iStream)%uTempFile, FILE=trim(TempFileName),ACTION='READWRITE',ACCESS='STREAM',ASYNCHRONOUS='YES',FORM='UNFORMATTED',STATUS='REPLACE',DISPOSE='DELETE')

After writing/reading/writing... the file then after close on this(these) unit(s), the temporary file(s) remains.

I have not tried using STATUS='SCRATCH', and my requirement is to not to use a scratch file as these (plural) are placed in a single directory specified by an environment variable. Multiple temporary files are created and when multiple physical drives are available, the temp files would like to be able to be distributed amoungst a specified list of devices.

The open(s) and close(s) performed with IFORT from:

w_HPCKit_p_2022.1.2.116_offline

*** However, the build log shows:

Compiling with Intel® Fortran Compiler Classic 2021.5.0 [Intel(R) 64]

and additional static libraries compiled with IFX show

Compiling with Intel® Fortran Compiler 2022.0.2 [Intel(R) 64]

MS VS Help | About shows (related to Fortran)

Intel® Fortran Compiler Package ID: w_oneAPI_2022.0.0.161
Intel® Fortran Compiler – toolkit version: 2022.1.0, extension version 22.0.0065.16, Package ID: w_oneAPI_2022.0.0.161, Copyright © 2002-2021 Intel Corporation. All rights reserved.

This version numbering confusion was posted on a different thread.

Jim Dempsey

[andrew_4619]

Hmm OK what if you close the file with status='delete' . At least that is standard unlike "dispose". And yes it should work shouldn't it.  FYI I recall Dr Fortran saying trim is unnecessary for the file= variable because it has to do that anyway.

[JohnNichols]

That has been happening to me, I used another method, but it was annoying. 

[Ron_Green]

Jim, I can't reproduce this.

At first I tried on my Linux system.  No luck, the file was deleted.

I tried Windows: still no luck.  

Just for sanity, I put in an ugly 'pause' statement and watched a File Explorer to confirm that the file was created, and deleted on CLOSE.  Can you try this simple little code, also I attached the code.  Try with and without PAUSE.  What am I missing?

program garbageDisposer
implicit none
real :: arr(100)=0.0
integer :: mySpud

open(newunit=mySpud,file='ohPleaseDeleteMe',ACTION='READWRITE',status='REPLACE',ACCESS='STREAM',ASYNCHRONOUS='YES',FORM='UNFORMATTED',DISPOSE='DELETE')
write(mySpud) arr
rewind(mySpud)
read(mySpud) arr
rewind(mySpud)
write(mySpud) arr
pause
close(mySpud)
end program garbageDisposer

[jimdempseyatthecove]

Hi Ron,

I will try your reproducer.

On my site the files remain after close.

The only difference that I can see (beyond file name) is (due to asynchronous i/o) I have wait(mySpud) sometime after the write (intervening code) but before the rewind or close.

Jim Dempsey

[jimdempseyatthecove]

That test file succeeds both IFX and IFORT with/without wait(mySpud)

Perhaps I am stopping the debug session before the close(units)?

Jim 

[Ron_Green]

So your usage is to run the code in the VS Debugger? 

Wondering if the debugger hits END PROGRAM and immediately wipes out the process image before the FRTL CLOSE is completed, either implicitly closed or explicitly closed.  CLOSE should not be asynchronous so I'd like to think that the CLOSE completes, the END PROGRAM is hit, and the VS debugger closes the session when the process image completes.   One would hope.    But if the code hits END or STOP without hitting a CLOSE statement, and implicit closing is occurring, the VS debugger may be overly aggressive in killing off the process image before the FRTL completes.  I have seen something similar with linux MPI daemons in the past shutting off the master rank too quickly before it's had a chance to clean up in the FRTL. 

Threading: you are not operating on this file from multiple threads are you, as in an OpenMP Parallel region(s)?

Probably stating the obvious, but if you do exit the debug session mid-execution it would leave the files, since CLOSE is yet to be called when the VS debugger closes the process image.

[jimdempseyatthecove]

I am convinced it was due to stopping the execution in the debugger prior to reaching the close/STOP/FRTL cleanup. The files are manipulated within parallel region. Input file sequentially read by multiple threads using same unit number, 4 temporary files per thread manipulated until a thread completion, then thread completions are joined as pairs (dropping one thread), and this repeats until one thread remaining, and at which point a single output file is written by one thread. After which (assuming I don't interrupt this) the file closes occur (which deletes the temporary files).

InputFile -> rotationally scatter blobs to threads -> threads process and reduce -> thread results cojoined, then process and reduce -> (repeat cojoining until one remaining thread) -> final process and write to output

Jim Dempsey

[jimdempseyatthecove]

Running in Debugger for now. Shaking out bugs, some are Heisenbugs.

Moved code over to system with Intel Parallel Studio XE 2020 IFORT. The oneAPI 2022 ICE (other post) does not occur there but some heisenbugs are showing up. In particular, (bounds checking on) I encounter a memory access error in a loop, substantial number of iterations in the loop, when examining the situation in the debugger, the array being accessed is undefined (yet it was defined for 1000's of iterations). The loop is only about 5 lines. Examining up a call level, the passed in array is valid, but going down to problem stack level the dummy looks/is undefined. Therefore, it became undefined somehow within this small loop. I suspect the stack argument containing the reference to the caller's array descriptor got wacked. Inserting sanity checks in the loop causes the bug to disappear. Haven't nailed the problem yet.

Jim Dempsey

[cryptogram]

I know that when you write to a file and close it,  the contents of the file are not always immediately available. To fix this, you

add a flush(unitnumber) right after the close, to make sure that the buffers get dumped into the file right away.

In my code I also delete files, and have been using the flush command for that as well. Not sure that it really does anything, but

the files seem to get deleted as expected, so it might be something to try.

open(887,file='exchange4.dat')
close(887,DISP='DELETE')
flush(887)

[Steve_Lionel]

No, FLUSH after CLOSE won't do anything useful.

DISP='DELETE' is handled by close operations in the run-time library, either from an explicit CLOSE (or STOP/ERROR STOP) statement or when called as part of the image "rundown". It could well be that if you stop execution in the debugger that this doesn't happen.

Note that DISP/DISPOSE is an extension, so its behavior is not specified by the standard.

[JohnNichols]

It would be nice if we had a command like C# ==> using () which can open flush and close a file.  

Often in C# the writing to the disk can be several seconds behind the program, there you have to use flush all the time.