Crash on file write

[Don_D_]

My program writes many small disk files, used for interprocess communication.  I have been doing this for years, but today things are weird.

Here's a section of my code:

        DO JV = JSTART,JEND,JSPAN
            
            CALL DERIVATIVES(JV,IVPSAVE,ABIG,JDRV)   ! GET DERIVS, THIS VARIABLES, ALL ABRNS
            IF (ISFLAGS(50) .EQ. 1) THEN        ! BOMB OUT NOW
                WRITE(MFILE,5001) -999     ! FLAG ABORT
                RETURN
            ENDIF
                  
            JFILE = 2000 + JV     ! EACH VARIABLE GETS ITS OWN FILE
            OPEN(JFILE,FORM='UNFORMATTED',STATUS='UNKNOWN',ACTION='WRITE',ERR=99) ! PUT RESULTS HERE
            REWIND(JFILE)
            LS = IAAN*(JV - 1) + 1
            LN = LS + IAAN - 1
            WRITE(JFILE) (DERIV(N),N=LS,LN)
        ENDDO

This works for JV = 1 and 2.  When it gets to 3, I get an "access violation reading ..." error at the WRITE(JFILE) line.  All of the data in this section are correct, and the array DERIV is filled with the right numbers.  Any idea what is going on?

[FortranFan]

Check the values of LS and LN for the case (JV=3) with the crash: either or both of these might be incorrect at the time, perhaps due to an erroneous value of IAAN or with respect to the array shape of DERIV?

[Don_D_]

Well, as I said, all data in this section are correct.  I checked everything.  So it's still a mystery.

But this brings up a related question:  What is the recommended way to get process A to send data to process B?  If a disk file with the data is not a good method, what is?

[FortranFan]

Don D. wrote:

.. What is the recommended way to get process A to send data to process B?  If a disk file with the data is not a good method, what is?

You may want to first short-list your options as well as preferences based on what's available in terms of your OS, this link may be of some help: https://docs.microsoft.com/en-us/windows/win32/ipc/interprocess-communications

Note with more recent versions of Intel Fortran compiler support toward Fortran standard facilities with interoperability with C, it's fairly straightforward to consume C and C-style C++ functions e.g., Microsoft Windows and SDK APIs.

So pipes (or sockets) might be an option here if FileMapping is unwieldy and links such as these may also be of help: https://docs.microsoft.com/en-us/windows/win32/procthread/creating-a-child-process-with-redirected-input-and-output?redirectedfrom=MSDN

[MWind2]

As you said all data is correct, perhaps by commenting out the Write and putting in a Print *,  IAAN, LS, LN; it could be a file share access problem. 

[Don_D_]

If I change the Write to a Print, will it write the data to the numbered file?  According to the manual, Print must never be used to transfer to user-specified IO units.

It should not be a file-sharing problem, since nobody else is reading or writing a file with that number at that time.

The comment by FortranFan is on the mark, however:

You may want to first short-list your options as well as preferences based on what's available in terms of your OS, this link may be of some help: https://docs.microsoft.com/en-us/windows/win32/ipc/interprocess-communic...

I have examined and printed many of the links there, and I think it will do the job.  But I have to ask: those are all C++ features.  Is there anything in native Fortran that can spawn a process, create a pipe, share memory with a child process, and the like?  Or is one forced to use C++?  It's a bag of worms, albeit powerful.

[MWind2]

I meant !comment out the write and use a print to see if any of those variables were out of bounds. If the number of jv is large, there may be a limit on open files, https://software.intel.com/en-us/forums/intel-visual-fortran-compiler-for-windows/topic/299178 was never publicly answered. In msvcrt it was 2048 if it's file open was used. I like memory mapped files for interprocess communication myself, but the use determines the best or even a good choice. 

[Don_D_]

The plot just got thicker.  I decided there were too many open files, so I am trying a different way.

Process 0 calls a C++ routine

         INTERFACE TO SUBROUTINE mymap[C,ALIAS:'_mymap'] ( DERIV )
             DOUBLE PRECISION DERIV [ REFERENCE ]
         END
       
        CALL mymap( DERIV(1) )

That routine sets up a CreateFileMapping and maps the array DERIV to a local array called *deriv.  So far, so good.  The other process sees the data put there by process 0.

What I want that other process to do is to return a pointer to DERIV so it will write to the same locations as process 0.  I got this far:

      POINTER DP
      TARGET DERIV
      
        INTERFACE TO SUBROUTINE mymap2[C,ALIAS:'_mymap2'] ( IP )
            INTEGER IP [REFERENCE]
        END
        
       DP => DERIV(1)
       IP = DP
       CALL mymap2( IP )       ! OTHER CORES MAP TO DERIV, I HOPE

mymap2 maps its own *deriv to the correct memory.  But I can't get it to return the address of the DERIV array for that memory, which is the same as that of deriv at this point, and should point to the array in process 0.

Perhaps I don't understand the use of pointers in Fortran.  In the code above, does DB => point to the absolute address of DERIV, or is it relative to its own address space?

How would you recommend I get a spawned Fortran program to use the same address space for this array as process 0?  I suspect it is simple, just obscure.

[Steve_Lionel]

DP points to wherever DERIV(1) lives. There are no relative addresses at play.

[Don_D_]

Well, that's what I thought (and expected).  But when I run

DP => DERIV(1)
       IP = DP
       CALL mymap2( IP )       ! OTHER CORES MAP TO DERIV, I HOPE

The integer IP is always zero.  Why isn't it the absolute address?

I can get the mapped address from a C++ routine, and it's not zero, and I can send it back to Fortran. But how do I get the latter to apply that address to the array DERIV?

Also, if I try

INTERFACE TO SUBROUTINE mymap2[C,ALIAS:'_mymap2'] ( DP )
            POINTER DP [REFERENCE]
        END

it won't compile.  Evidently a pointer cannot show up in an INTERFACE block.

Can you answer my previous question?  Maybe I am going about it all wrong.

How would you recommend I get a spawned Fortran program to use the same address space for this array as process 0?  I suspect it is simple, just obscure.

[Steve_Lionel]

Fortran pointers aren't like C pointers. When you do an intrinsic assignment (=) of a pointer, you're assigning the target, not the address. The only way in standard Fortran to get the address from a pointer is C_LOC and even then you have to use TRANSFER to cast it to an integer.

You're showing only tiny snippets of code, so it's hard to speculate as to what is going on,

Last, your INTERFACE TO is ancient Microsoft Fortran PowerStation syntax, and I don't think it's what you want anyway. Don't try passing the pointer as a pointer, unless your C code is willing to deal with a F2018 "C descriptor". Just declare it as whatever data type it is. Please use standard Fortran C interoperability features and syntax.

[Don_D_]

The underlying question is simple:  How can make a child Fortran program change numbers in the address space of the parent Fortran program?  What I have done is trial-and-error programming.  Is there a good way?

[Steve_Lionel]

What is the underlying problem you're trying to solve for which you've seized on this memory sharing mechanism? A much simpler approach is a module variable in a global read-write section in a DLL. This is data that can be shared among different executables. An example of this is the DLL_shared_data sample in the Intel Parallel Studio XE for Windows sample bundle.

Depending on the problem space, coarrays might be another option.

[FortranFan]

Don D. wrote:

The underlying question is simple:  How can make a child Fortran program change numbers in the address space of the parent Fortran program?  What I have done is trial-and-error programming.  Is there a good way?

Can the "child program" be redesigned to work as a Windows DLL? https://docs.microsoft.com/en-us/windows/win32/dlls/dynamic-link-libraries

Should such an option exist, you will find the desired data exchange with such a program design is rather straightforward.

If not, Pipes or File Mapping per the links I provided above are your best options.

And note this has nothing to do with Fortran: for your specific computing needs and environment, if you can narrow down how best a child C/C++ program can change numbers in the address space of the parent C/C++ program, you can replicate the same with your Fortran program.

[jimdempseyatthecove]

>>

That routine sets up a CreateFileMapping and maps the array DERIV to a local array called *deriv.  So far, so good.  The other process sees the data put there by process 0.

What I want that other process to do is to return a pointer to DERIV so it will write to the same locations as process 0.  I got this far:

      POINTER DP
      TARGET DERIV
      
        INTERFACE TO SUBROUTINE mymap2[C,ALIAS:'_mymap2'] ( IP )
            INTEGER IP [REFERENCE]
        END
        
       DP => DERIV(1)
       IP = DP
       CALL mymap2( IP )       ! OTHER CORES MAP TO DERIV, I HOPE

mymap2 maps its own *deriv to the correct memory.  But I can't get it to return the address of the DERIV array for that memory, which is the same as that of deriv at this point, and should point to the array in process 0.

<<

What you are trying to do is use a memory mapped file. This is fine, as long as the complete file can fit within available space...
... .AND. you realize that the virtual address of this file (buffer) in process A is not necessarily at the same virtual address of process B...
... .AND. "available space" is the amount of space specified in Windows with an upper limit set by Non Page Pool Size.

Instead of passing the virtual address of the item in process A to process B, pass the relative address from the base of the memory mapped file to process B (then B can add to this the base of its memory mapped file of same file).

The DLL_shared_data method that Steve Lionel mentions is an alternative (but may have limitations when the size of the shared data is not determined until runtime).

Because you appear to have memory mapped file "sortof" working, try the relative address hint first.

Jim Dempsey