forrtl severe 28 - close command problem

[grasso__francesco]

dear all,

i wrote a code 2 years ago in which data are read from text file and then the file is deleted during the close command. Since then the code Always worked until this morning when i started to get Runtime error in the title, where i cannot access the file and so it cannot be deleted.

below an example of the original code:

open(unit=10, file=filename)

read(10, *) data

close(10, status='delete')

i found already a solution to the practical problem, which is just closing the file and then after using "system" command to delete the file. It works but it is much slower than the original solution. So i would really like to go back to the original commands!!

any clue about what could i do or what got wrong all of sudden? according to the info online it seems it is some conflicts between the OS (Windows 8) and the code...but i used it as it is since 2 years and it Always worked...

thanks in advance for help

francesco

[andrew_4619]

Where is the file located. Windows uac protects some locations.....

[grasso__francesco]

hi Andrew,

the file is located inside the general folder created by Visual Fortran for the projects.

I tried to copy the executable on different folder and then on different PCs. on WIN7 it works fine, but on WIN8.1 (same as mine) it produces the same problem...

[jimdempseyatthecove]

Does this help: https://support.microsoft.com/en-us/help/2623670/-access-denied-or-other-errors-when-you-access-or-work-with-files-and

Look at issue 2

Google: windows 8.1 delete file access denied

for additional information

Jim Dempsey

[grasso__francesco]

hi Jim,

i tried your option and i followed the guidelines on microsoft site but it does not change.

I also tried to run troubleshoot compatibility but no change

tanks anyway

Francesco 

[jimdempseyatthecove]

There are some peculiarities with the Windows file system that were introduced in the more recent versions of Windows.

One possible cause, which is completely a guess, is a file cannot be deleted if it is open for use (either other process or other file handle in same process). As to what can cause this...

a) Anti-Virus program is scanning the file (usually just after file creation or updated). You can turn off AV for selected folder and/or selected files)
b) When Windows file indexing is enabled for the folder in which the file exist, and the file is being indexed. (You can disable file indexing for selected folders or for selected device)
c) When the file is being cached by the Windows file system (in anticipation of it being read to the end.

There may be other circumstances

Jim Dempsey

[andrew_4619]

use ifwin, only:  sleep
integer :: istat

.....
.....

close(10, status='delete', iostat = istat)
if ( istat /= 0 ) then
  call sleep(500) ! wait for 500 mille seconds
  close(10, status='delete', iostat = istat)
endif

Try something like the above to check if it is not a windows problem with the file being finished with by fortran but not by windows so still being 'locked'. 

[grasso__francesco]

thanks all for help.. however the issue is still there

@Andrew: I tried your idea and the code does not crash anymore. However the file is not deleted and as consequence, the new data are appended in the same file, resulting in the software not working properly because does not read the right lines.

btw: the function sleep seems to count seconds and not milliseconds, while sleepqq uses milliseconds. not sure about the difference between the two..

@JIm: thanks for the options. Since i work with a company laptop, I required administrator rights to access AV settings and OS settings. What exactly I should check to verify if one of yuor options is the source of my issue?

Francesco

[jimdempseyatthecove]

>>What exactly I should check to verify if one of your options is the source of my issue?

Right-Click on the folder containing the data file, select Properties. On General tab click on Advanced... uncheck all boxes. (You can experiment checking boxes to desired state after you determine cause of issue).

>>However the file is not deleted and as consequence, the new data are appended in the same file

On the OPEN statement you can add STATUS='REPLACE'

------- IVF documentation --------

OPEN: STATUS Specifier

The STATUS specifier indicates the status of a file when it is opened. It takes the following form:

STATUS = sta

sta

Is a scalar default character expression that evaluates to one of the following values: 'OLD' Indicates an existing file. 'NEW' Indicates a new file; if the file already exists, an error occurs. Once the file is created, its status changes to 'OLD'. 'SCRATCH' Indicates a new file that is unnamed (called a scratch file). When the file is closed or the program terminates, the scratch file is deleted. 'REPLACE' Indicates the file replaces another. If the file to be replaced exists, it is deleted and a new file is created with the same name. If the file to be replaced does not exist, a new file is created and its status changes to 'OLD'. 'UNKNOWN' Indicates the file may or may not exist. If the file does not exist, a new file is created and its status changes to 'OLD'.

Scratch files go into a temporary directory and are visible while they are open. Scratch files are deleted when the unit is closed or when the program terminates normally, whichever occurs first.

To specify the path for scratch files, you can use one of the following environment variables:

• On Windows*: FORT_TMPDIR, TMP, or TEMP, searched in that order

• On Linux* and OS X*: FORT_TMPDIR or TMPDIR, searched in that order

If no environment variable is defined, the default is the current directory.

The default is 'UNKNOWN'. This is also the default if you implicitly open a file by using WRITE. However, if you implicitly open a file using READ, the default is 'OLD'. If you specify compiler option f66 (or OPTIONS/NOF77), the default is 'NEW'.

Jim Dempsey

[grasso__francesco]

Jim,

i followed your procedure about folder properties and the only box ticked was the one about indexing, which is now off.

About your proposal to put 'replace' in open statement, it does not work. The fact is that my software runs an external executable which generates iteratively the file i want to read. Putting 'replace' attribute in my software would result in deleting the file at the opening moment, which will prevent me to first read the data inside.

Perhaps the external executable could contain the replace attribute, but i dont have the source files and the way is programmed is to actually append new data in case the file is already existing. This is actually why i need to make sure the file is deleted before the external executable is called again in the next iteration.

Francesco

[jimdempseyatthecove]

OPEN(UNIT=10, FILENAME='ASDF.DAT', STATUS='OLD')
READ(10,*) DATA
CLOSE(10)
ISTATUS = RENAME('ASDF.DAT','C:\Windows\Temp\ASDF.DAT.tmp')

if that has issues, then

USE IFPORT
...
OPEN(UNIT=10, FILENAME='ASDF.DAT', STATUS='OLD')
READ(10,*) DATA
CLOSE(10)
ISTATUS = RENAMEFILEQQ('ASDF.DAT','C:\Windows\Temp\ASDF.DAT.tmp')
IF(ISTATUS /= 0) THEN
  IERROR = GETLASTERRORQQ()
  ! See IVF IFPORT.F90 for error codes
  ...
ENDIF

Jim Dempsey

[jimdempseyatthecove]

Note, This is Fortran. Do not use \\ in file spec in quoted string.

Jim Dempsey

[gib]

Jim, doesn't this result in a problem the next time the program is run, renaming the file to the name of an existing file?  Assuming it's even possible to rename a file changing its directory.

[jimdempseyatthecove]

>>Jim, doesn't this result in a problem the next time the program is run, renaming the file to the name of an existing file?

Presumably, if delete fails, then the RENAMEFILEQQ will also fail, however RENAMEFILEQQ will leave information for GETLASTERRORQQ, which you can use to determine what is the underlying cause. The two leading causes are:

Protection violation (e.g. file is protected against Delete or Write)
File in use (either your application has it open on different unit, or some other application has it open).

However there can be unexpected other causes. When you know the cause, you may be able to code around this, that failing you can fall back to using SYSTEMQQ to delete the file. However note that SYSTEMQQ with a command to delete the file may return while the delete is in progress and then if your program creates a new (same named - same place) file you may experience the same issue (*** see note below)

The rename conflict (assuming renaming works) can be corrected by using a unique file name (e.g. yymmddhhmmssmsus.DeleteMe)

You can place the renamed file either in the same folder that the original file resided in or place into the WindowsTemp folder. Wherever you place the resultant file, you can have a background process running that sleeps for a period of time, then attempts to delete any/all of the *.DeleteMe files. E.g. after the AV gets done scanning the file or whatever also has the file open closes it. Note, Andrew's suggestion in #7 tried to address the AV issue by having your program retry after 0.5 seconds. If AV was the cause, and your file takes longer than 0.5 seconds to scan, then the fix would be to extend the delay time to just longer than the worst case scan time (or repeatedly attempt to delete the file, possibly with a "Failed to delete after nn tries. Continue or Abort?"

*** Note (afterthought)

The 3rd party utility that creates the file, and which you cannot fix/edit, may create the file in the "Current Directory". If that be the case then (sketch)

    MyDir = FILE$CURDRIVE
    dirLen = GETDRIVEDIRQQ(MyDir)
    if(dirLen == 0) stop ! something wrong
loop:
    newTempDir = MakeNextSequenceTempDir() ! e.g. 'yymmddhhmmssmsus'
    iStat = MAKEDIRQQ(newTempDir)
    ... ! error test
    iStat = CHANGEDIRQQ(newTempDir)
   ... ! error test
   Call The3rdPartySubroutineToCreateFile()
   iStat = CHANGEDIRQQ(MyDir)
   TheFile = newTempDir // '\' // TheDefaultFileName
   OPEN(10,FILE=TheFile)

Then after program runs you can cleanup the disk. (or attempt to cleanup in background process)

Jim Dempsey

[gib]

Jim, I am not Francesco, I'm just an interested onlooker.

By the way Windows does not permit renaming of a file to a different directory in Explorer, so I'm guessing that it doesn't allow it anywhere.

Cheers

Gib

[jimdempseyatthecove]

https://msdn.microsoft.com/en-us/library/windows/desktop/aa365239(v=vs.85).aspx

BOOL WINAPI MoveFile(
  _In_ LPCTSTR lpExistingFileName,
  _In_ LPCTSTR lpNewFileName
);

Jim Dempsey

[Eugene_E_Intel]

On a related note, you can find, which process has a particular file open, using the Process Explorer tool from Microsoft:

https://docs.microsoft.com/en-us/sysinternals/downloads/process-explorer

To use, install the tool, open it, and search (probably CTRL-F) for the file name.  For best results, you need to run this tool with admin privileges.

[grasso__francesco]

hi all,

i solved the problem!! at the end, as suggested also in this forum, the AV was blocking the software in accessing the files to delete them!! I had to uninstall and install again the AV and now is working with the original 'delete' instruction in the CLOSE command.

however, the AV is still trying to scan all the files generated by my code and so the overall computational time is increased 4 times. So now i am trying to convince IT people to create a exclusion folder so that the AV does not bother me

thanks again for helping

Francesco