OpenMP problem

[Gib_B_]

My Fortran program is built as a DLL, and it calls a C function that is also built as a DLL.  It is all working fine.  I have now started trying to parallelize a critical loop using OpenMP.  The C function no longer works correctly when it is called from an OpenMP thread (it fails with an error message that is not appropriate for the arguments supplied - the same set of arguments gives no trouble in single-thread mode).  This is a sketch of what I am doing:

!$omp parallel do private(a,...)

do i = 1,n

    a = ...

    call fortran_sub(a,...)

enddo

!$omp end parallel do

...

subroutine fortran_sub(a,...)

    call c_function(a,...)

end subroutine

Since all the arguments of fortran_sub() are private, it is my understanding that c_function() should execute happily in separate threads.  Something is going wrong, however, and I have no idea what it is.  I've used OpenMP in several other programs, but never calling a C function within the loop.  The C compiler is MSVS 2005, the Fortran is IVF 11.

[IanH]

Does c_function define (or whatever the appropriate C terminology is) any static variables - i.e. does it have internal state that might be shared between threads?

Is your C function compiled with the multi-threaded runtime library selected?  (I can't remember whether VS2005 still offered the single threaded runtime option.)

Is stack exhaustion a possibility?

[Gib_B_]

Hi Ian,

I do not think there are any static "state" variables, but I'm not intimately familiar with this code, which I did not write.  I doubt that stack exhaustion could be the cause - the data requirements of the function are very small, and it goes bad quickly (although not immediately).  The C DLL is built as Multi-threaded DLL.

I need to check out your first two points carefully.

Thanks

Gib

[Gib_B_]

The plot thickens!  There is no problem when the programs are built on a machine using MSVS 2012 and Intel Composer XE 2013.  This is a clue - what does it suggest?

[Gib_B_]

Update: I copied the .dll and .lib built with MSVS 2012 to the other machine, and it works fine in single-thread mode.  It fails in the same way in multi-thread mode, suggesting that the problem is not caused by the C side.  That leaves MSVS 2005 or IVF 11.0.075 as the possible culprit.

[IanH]

11.0.075 is a bit old.

How are you describing the interface of the C function to the Fortran?

Mixing C runtimes like that will cause issues one day (actually.... are they consistent C/Fortran with your VS2005 setup?)

[Gib_B_]

Yes, it is old.  At one stage a few years ago, as a result of some issue that I can't now recall, an Intel person offered me a more recent version, 11.1.x I think, but I was never successful in following the steps required to download it, despite multiple attempts.  I eventually gave up.

Here is the interface code:

interface

    subroutine min_dist(a1,b1,centre1,orient1,a2,b2,centre2,orient2,tol,s1,s2,r1,r2,d,res) BIND(C,NAME='min_dist')
    use :: iso_c_binding
    real(c_double),value :: a1, b1, a2, b2, tol
    real(c_double) :: centre1(*), orient1(*), centre2(*), orient2(*)
    real(c_double) :: s1, s2, r1, r2, d
    integer(c_int) :: res
    end subroutine

end interface

I don't follow your comment about mixing runtimes - I thought Fortran-C interoperability was a done deal.  I've never encountered problems mixing Fortran and C with MSVS 2005 before, although usually I call Fortran DLL subroutines from C/C++.  What do you mean "are they consistent C/Fortran with your VS2005 setup?"?  I use both IVF 11.0.075 and MS C/C++ from within MSVS 2005, if that is your question.

[IanH]

I'm perhaps just repeating an earlier point.  You generally don't want more than one instance of the various runtime libraries in your application.  This includes the C runtime, the Fortran runtime and the OpenMP runtime (particularly that last one, given its an OpenMP problem and I think OpenMP runtime library selection options were different back then).

If you have a DLL built using VS2012 it will be using the VS2012 runtime.  If your application has been built on VS2005, it will be using the 2005 runtime.  They will fight at some point over things like memory allocation or input/output, unless you are careful to avoid things "crossing the boundary" of the respective run times (e.g. a file opened with one runtime being written to by another is bad).

Generaly if you have an application with DLL's where there might be things "crossing the boundary" then you need the DLL version of the runtimes in every DLL and in the EXE.  If you C runtime is multithreaded DLL your Fortran runtime needs to be the same.  That's a default that has changed with ifort version.  There was a similar setting for OpenMP runtime that might have been relevant around ifort 11.0.

But I'm obviously just guessing.

[jimdempseyatthecove]

In the newest version of Fortran there is a distinction between what VALUE means with respect to Fortran calling convention and value means with C/C++ calling convention. For VALUE, the latest Fortran creates a temp variable (copy) then passes a reference to the copy. Even mixing versions of Fortran can expose interoperability issues.

Jim Dempsey

[IanH]

What do you mean by "version of Fortran"?  Do you mean "version of ifort"?

[jimdempseyatthecove]

IanH,

Steve may have to answer this, but I think the new standard requires a reference to a newly created temporary. This was done to resolve issues between optional arguments and present() where the optional argument is value and the value could contain 0.

Jim Dempsey

[Gib_B_]

By "version of Fortran" I mean version of Intel Visual Fortran: 11.0.075.

To respond to Jim, in case you didn't catch it: the problem occurs when using Fortran (IVF 11.0) and C code built using MSVS 2005.  The problem goes away when I use Intel Composer XE 2013 and MSVS 2012, and, surprisingly, the C DLL built with MSVS 2012 works correctly with the old compiler using MSVS 2005.

Steve?

[Steven_L_Intel1]

No clue - sorry.

[IanH]

jimdempseyatthecove wrote:
Steve may have to answer this, but I think the new standard requires a reference to a newly created temporary. This was done to resolve issues between optional arguments and present() where the optional argument is value and the value could contain 0.

The standard doesn't talk at that level of implementation detail.  Aspects of ifort's implementation changed, perhaps to accommodate VALUE + OPTIONAL,  but they had other choices.

[Steven_L_Intel1]

For a BIND(C) routine, the VALUE attribute simply means pass by value - no temporary is created. We've always done this and the standard specifies it (though not in those words.)

For a routine without BIND(C), we used to have the VALUE attribute mean pass-by-value, where the standard said pass a temporary copy by reference (again, not in those words). We do this now if /standard-semantics is used but in version 15 it will be the default.

We don't need to change anything for VALUE + OPTIONAL, which is disallowed in F2008 but is allowed in the interoperability extensions TS29113 (and Fortran 2015). VALUE + OPTIONAL is not allowed for BIND(C) routines regardless. I know at least one other vendor that did need to change their implementation because of this.