OpenMP problem

[fobermaier]

Hello,
I'm trying to tune an existing application with help of OpenMP.
Everything works fine for a while but then I get an access violation inside a function that lies in a DLL. Is that a problem?
Do !$OMP CRITICAL/BARRIER etc. directives inside the code of the DLL matter?

Both application and DLL are compiled with IVF 10.1

Thanks in advance
FObermaier

[Steve_Nuchia]

I would hope that would work, but it may work only if both your DLL and the code that calls it share the same OMP library. If either or both are statically linked to the runtime I could see that causing a problem.

I would never have tried spreading the concurrency control across a DLL boundary on purpose, but I'm kinda conservative that way. And one does not always have control of these things.

[jimdempseyatthecove]

FObermaier,

!$OMP CRITICAL
!$OMP CRITICAL(YouPickANameHere)

Think about this and you can answer the question for yourself.

A DLL does not link to addresses contained within the application that calls the DLL. An application written with OpenMP enabled uses either a default mutex or a named mutex for the critical section. This (these)mutex (mutii??)is (are)contained within the application's statically linked data and are not visible to the DLL. The DLL with OpenMP when linked will instantiate its own critical section(s) OR the DLL will get a placeholder for an extern symbol (that of the default or named mutex) but who's address is never inserted into the DLL(probably remains NULL). The compiler writer could tell you what is happening.

The followingwill correct for this (Intel may have a better suggestion)

On calls to the DLL that will (may)require the use of a critical section, pass in an additional argument that contains the address of a subroutine in the applicationthat is called with a flag as a call-back routine (1 = enter critical section, 0 = exit critical section). Then the DLL uses the call-back routine to enter YourCallbackFuncition(1) and exit YourCallbackFuncition(0) the critical section.

I would suggest against having any !$OMP directives in the DLL as it will make sharing the DLL difficult. If you cannot extract the !$OMP directives convert the DLL back to a static library.

Jim Dempsey

[fobermaier]

Hello,

thanks for the insight and suggestions.

I converted the DLL to as static library and doing that, I've had partial success: the first part works now.
The main task -which does not work- is this:

      type(T_ISN), pointer :: pISN
      type(T_ISB_V3), pointer ::  pISB
      integer(4)              ::  numIFBs
      type(T_IFB_V3), pointer :: pIFBs(:)
      type(T_ISB_V3), pointer :: pIZB
      interface
        function fnPLT(pIFBL, value, lt, l, t) result(dres)
          use ModuleIFB, only:T_IFB_Sub
          type(T_IFB_Sub), pointer :: pIFBL(:)
          real(4), intent(in)         ::  value
          integer(4), intent(in) ::  lt
          integer(4), intent(out) ::  l, t
          real(8)  ::  dRes
        end function
      end interface
      interface
        subroutine subFactor( factor, factorMin, factorMax, numFactors, sumFactors )
          real(8), intent(in)  ::  factor, factorMin, factorMax
          integer(4), intent(inout)  ::  numFactors
          real(8), intent(inout)     :: sumFactors
        end subroutine
      end interface
      interface 
        subroutine subDoNext( pMat, fileName, daOrig, daDest )
          use V3Mat, only: T_MATRIX
          type(T_MATRIX), pointer :: pMat
          character(len=*) ::  fileName(:)
          real(8) :: daOrig(pMat%iOdCount), daDest(pMat%iOdCount)
        end subroutine
      end interface
      real(8), intent(in)  ::  validRange(2)

      integer(4) ::  i
      type(T_IFB_V3), pointer :: pIFB

      !$OMP PARALLEL DO DEFAULT(PRIVATE) LASTPRIVATE(i) SHARED(numIFBs, pISN, pIFBs, pISB, pIZB, validRange) !!!!, fnPLT, subFactor, subDoNext)
      do i = 1, numIFBs
        pIFB => pIFBs(i)
        call TestIFBToISB( pISN, pIFB, pISB, pIZB, fnPLT, subFactor, subDoNext, validRange )
      end do
      !$OMP END PARALLEL DO

The errors I get range from access-violation invalid index.
Calling the subroutine TestIFBToISB may take up 7 minutes, depending on the data that is beeing tested. Is this a legitimate approach with OpenMP or did I miss something.
The UDT-Pointer arguments only supply data, their values are not changed within TestIFBToISB or the functions/subroutines it calls. 

Thanks in advance
FObermaier

[jimdempseyatthecove]

Do you have a type-o on your !$OMP PARALLEL DO...

SHARED(numIFBs, pISN, pIFBs, pISB, pIZB, validRange) !!!!, fnPLT, subFactor, subDoNext)

Your loop has DEFAULT(PRIVATE) but the arguments are not used/defined??

Jim Dempsey

[fobermaier]

I understood the

DEFAULT( mode )

directive is there so one does not have to list all the variables which are to be treated (PRIVATE/SHARED).

The directive evolved (since I'm new to OpenMP), thats why I didn't write

!$OMP PARALLEL DO DEFAULT(SHARED) FIRSTPRIVATE(i) PRIVATE(pIFB)

FObermaier

[jimdempseyatthecove]

FObermaier,

>> !$OMP PARALLEL DO DEFAULT(SHARED) FIRSTPRIVATE(i) PRIVATE(pIFB)

Then, with the above, do you still get the access error?

If so, then step into TestIFBToISB (you may have to execute 1 statement in the subroutine) then attempt to examine each of the entry arguments. See which have problems being examined or have junk. This may point to a problem further up in the call tree. Note, if TestIFBToISB does not have an interface then you may be passing incorrect arguments.

Jim Dempsey

[fobermaier]

Jim Dempsey,

the !$OMP PARALLEL DO ... clause was not the problem. Subroutines called within TestIFBToISB deallocated/nullified/dereferenced local pointers they pass to other subroutines/functions. Also I missed,
that some variables were declared using 'save'. Now it works and even produces the same results as the
single threaded version.

In my opinion the IVF documentation concerning OpenMP lacks information on possible pitfalls using OpenMP.
At least I couldn't find them.

Thanks
FObermaier

[TimP]

Much of the OpenMP syntax checking is reserved to Intel Thread Checker. The general approach of OpenMP is to trust the programmer until proven wrong with a specific test case.
I've found web searches useful on OpenMP topics, along with the Chapman, Jost & van der Pas textbook. When you go beyond what is well tested, you take on a research project of your own.

[jimdempseyatthecove]

FObermaier,

Glad to see you found the problem and can move on.

Some improvement to the documentation would be welcomed. Especially in areas of pitfalls. Too often the author of the documentation is skilled at the art of the programming being documented. It is hard for them to keep in mind the difference between "Nobody would be stupid enough to do..." verses "A beginning programmer in this art form might fall into the trap of...". Writing good documentation is often more difficult than writing a good program.

Jim Dempsey