Explicit Interface Fails with C Binding

[SunMesa]

Hello all,

The following will describe a scenario/associated code that is much simpler than the real one, but it will serve to illustrate the problem.

Suppose I have a Fortran DLL that is essentially a black box (source code/author unavailable), but that contains certain desired functions with known interfaces that we would like to access from a driver application that loads this DLL.

Initially, let's say that this DLL contains the following components of interest:

      Integer Function F1(X,N)
      USE NMod
      IMPLICIT NONE
      !DIR$ ATTRIBUTES DLLEXPORT :: F1
      Real, Intent(IN) :: X
      Integer, Intent(OUT) :: N
      N = 1 + FLOOR(10*ABS(SIN(X)))
      F1 = N**2
      NShared = N
      End Function F1

      Integer Function F2(Y,Z)
      USE NMod, N=>NShared
      IMPLICIT NONE
      !DIR$ ATTRIBUTES DLLEXPORT :: F2
      Real, INTENT(IN) :: Y
      Real, INTENT(OUT) :: Z(N)
      Integer :: i
      Do i = 1,N
        Z(i) = Y + i
      EndDo
      F2 = INT(SUM(Z))
      End Function F2

      MODULE NMod
        Integer :: NShared
      END MODULE NMod

and then suppose we have a Fortran driver that links to this DLL, as follows:

      Program TestDriver
      IMPLICIT NONE
      Integer :: F1,F2
      Integer :: iF1,iF2,N
      Real, Allocatable :: Z(:)
      Real :: X,Y
      CALL RANDOM_SEED()
      Call RANDOM_NUMBER(Y)
      X = 10**Y
      iF1 = F1(X,N)
      Allocate(Z(N))
      iF2 = F2(Y,Z)
      Write(*,*)'iF1+iF2=',iF1+iF2
      Deallocate(Z)
      End Program TestDriver

Note that the DLL function F2 is getting the parameter N via the renaming of NShared from Module NMod, with NShared being defined in function F1.

At least in my Visual Studio 15.9.21 / Intel Fortran 19.1 environment, the above DLL will build with no complaint, and the driver will similarly build and execute with no errors.

Now add a new wrinkle... suppose that the original DLL was designed with C-interoperability, as follows:

      Integer(C_INT) Function F1(X,N) Bind(C)
      USE NMod
      USE, INTRINSIC :: ISO_C_BINDING
      IMPLICIT NONE
      !DIR$ ATTRIBUTES DLLEXPORT :: F1
      Real(C_FLOAT), Intent(IN) :: X
      Integer(C_INT), Intent(OUT) :: N
      N = 1 + FLOOR(10*ABS(SIN(X)))
      F1 = N**2
      NShared = N
      End Function F1

      Integer(C_INT) Function F2(Y,Z) Bind(C)
      USE NMod, N=>NShared
      USE, INTRINSIC :: ISO_C_BINDING
      IMPLICIT NONE
      !DIR$ ATTRIBUTES DLLEXPORT :: F2
      Real(C_FLOAT), INTENT(IN) :: Y
      Real(C_FLOAT), INTENT(OUT) :: Z(N)
      Integer :: i
      Do i = 1,N
        Z(i) = Y + i
      EndDo
      F2 = INT(SUM(Z))
      End Function F2

      MODULE NMod
        Integer :: NShared
      END MODULE NMod

This DLL will also build with no complaints. BUT, when building the driver application above linked to this new DLL, both F1 and F2 are called out as unresolved externals, apparently as a result of the C binding.

The first inclination is to replace the simple "Integer :: F1,F2" in the driver with explicit interfaces, as follows:

      Program TestDriver
      IMPLICIT NONE
      INTERFACE
        Integer(C_INT) Function F1(X,N) Bind(C)
        USE, INTRINSIC :: ISO_C_BINDING
        IMPLICIT NONE
        Real(C_FLOAT), Intent(IN) :: X
        Integer(C_INT), Intent(OUT) :: N
        End Function F1
      END INTERFACE
      INTERFACE
        Integer(C_INT) Function F2(Y,Z) Bind(C)
        USE, INTRINSIC :: ISO_C_BINDING
        IMPLICIT NONE
        Real(C_FLOAT), INTENT(IN) :: Y
        Real(C_FLOAT), INTENT(OUT) :: Z(N)
        End Function F2
      END INTERFACE
      Real, Allocatable :: Z(:)
      Real :: X,Y
      Integer :: iF1,iF2,N
      CALL RANDOM_SEED()
      Call RANDOM_NUMBER(Y)
      X = 10**Y
      iF1 = F1(X,N)
      Allocate(Z(N))
      iF2 = F2(Y,Z)
      Write(*,*)'iF1+iF2=',iF1+iF2
      Deallocate(Z)
      End Program TestDriver

This results in some progress... F1 and F2 are no longer unresolved externals, but now there is grief from the compiler about function F2. The INTERFACE block for function F2 in TestDriver, specifically the statement "Real(C_FLOAT), INTENT(OUT) :: Z(N)", is called out for:

"error #6279: A specification expression object must be a dummy argument, a COMMON block object, or an object accessible through host or use association. [N]"

as well as

"warning #8586: Implicit type is given to allow out-of-order declaration. Non-standard extension. [N]".

So my fundamental questions at this point are:

(1) The compiler evidently thinks there are no ambiguities or conflicts in the pure Fortran version of the TestDriver/TestDLL package... it builds and executes with no problem. Why does it balk at the C-bound equivalent?

(2) Assuming that the C-bound DLL is immutable, is there a work-around, either in the compiler settings or the driver source code, that will allow the TestDriver and the (C-bound)TestDLL to build and execute?

Thanks so much for any guidance!

[IanH]

I assume the source defining the module is provided to the compiler ahead of the source referencing the module.  (If not, you should switch the program unit order in your dll source to put the module first.)

The interface block for f2 in the driver needs to use the module that defines N.  In that interface block you are missing...

USE NMod, N=>NShared

The language reason that the USE is required is that NShared (via the name N) is part of the definition of the interface (it specifies the size of the array dummy) - so you need to tell the compiler what N is.  This is really just the same as using a variable name that hasn't been specified by a type declaration statement when implicit none is in use.

Depending on compiler implementation (this bit is not a language question), the executable may need to see the value of NShared from the DLL at the time that it sets up the call to F2.  You should modify the DLL code to export that symbol...

      MODULE NMod
      IMPLICIT NONE
      !DIR$ ATTRIBUTES DLLEXPORT :: NShared
      Integer :: NShared
      END MODULE NMod

(Testing here shows that the 2021.4 version of the compiler with defaultish options doesn't reference the value of NShared from the DLL, but it might change its mind one day.)

Previously everything was being done with implicit interfaces.  When compiling the previous driver, the compiler really has no idea whether there were ambiguities or conflicts - it just assumes (following the rules of the language) based on the information that is implicitly provided by the syntax of the function references in the driver source. 

Interoperable procedures require an explicit interface at their point of reference - at a simple level so the compiler actually knows that the procedure being referenced is interoperable (it may need to set the call to the function up differently). 

[SunMesa]

@IanH

Thanks for your very informative reply!

I assume the source defining the module is provided to the compiler ahead of the source referencing the module.  (If not, you should switch the program unit order in your dll source to put the module first.)

I'm honestly not sure in what order the compiler sees the module. My TestDLL project with the C-binding has three distinct files in the Visual Studio "Source Files" folder, listed in alphabetical order, namely Func_F1_CBind.f90, Func_F2_CBind.f90, and Module_NMod.f90.

---

The interface block for f2 in the driver needs to use the module that defines N.  In that interface block you are missing...

USE NMod, N=>NShared

I actually had tried that previously, inserting it into the F2 interface block precisely as it appears in the DLL source code. Attempting to compile the driver then gave the following errors:

error #7002: Error in opening the compiled module file. Check INCLUDE paths. [NMOD]
error #6223: A specification expression is invalid. [N]
error #6498: The use-name for this local-name is not defined. [N]

Although I can't edit the real-world DLL I'm working with, I will try adding the !DIR$ ATTRIBUTES DLLEXPORT :: NShared in the NMod module in my TestDLL as per your suggestion, and report back on my results. Thanks again for your reply-

[FortranFan · Accepted solution]

@SunMesa writes:

.. my fundamental questions at this point are:

(1) The compiler evidently thinks there are no ambiguities or conflicts in the pure Fortran version of the TestDriver/TestDLL package... it builds and executes with no problem. Why does it balk at the C-bound equivalent?

(2) Assuming that the C-bound DLL is immutable, is there a work-around, either in the compiler settings or the driver source code, that will allow the TestDriver and the (C-bound)TestDLL to build and execute?

Thanks so much for any guidance!

@SunMesa ,

Re: your question 1), you have the explanation by @IanH.

Re: question 2), please note you only need to use the facilities in Fortran starting Fortran 2003 toward interoperability with C if you know this "blackbox" DLL you are working with is indeed based on a C processor, either directly or via Fortran with bind(C) or similar extensions, or extern "C" with C++, etc.

Assuming the blackbox DLL is C interoperable, a variant option I suggest is you introduce your "integration" module to interface your code with this DLL, say like so:

      module Ibb_m
      ! Module to integrate your code with a blackbox DLL

         integer :: N   !<-- Enclose N as a module entity
         INTERFACE
           Integer(C_INT) Function F1(X,N) Bind(C, name="F1") !<-- Note the name clause
           USE, INTRINSIC :: ISO_C_BINDING
           IMPLICIT NONE
           Real(C_FLOAT), Intent(IN) :: X
           Integer(C_INT), Intent(OUT) :: N
           End Function F1
         END INTERFACE
         INTERFACE
           Integer(C_INT) Function F2(Y,Z) Bind(C, name="F2") !<-- Note the name clause
           USE, INTRINSIC :: ISO_C_BINDING
           import :: N   !<-- Note this statement to import N from the host
           IMPLICIT NONE
           Real(C_FLOAT), INTENT(IN) :: Y
           Real(C_FLOAT), INTENT(OUT) :: Z(N)
           End Function F2
         END INTERFACE

      end module 

Now, given your original post conveys the impression you have access to the blackbox DLL and also a mechanism to link to it, perhaps using an import library.  Say for illustration purposes they are named 'bb.dll' and 'bb.lib' respectively.  You can then work with what you have like so:

      Program TestDriver
         use Ibb_m, only : F1, F2, N !<-- Note the explicit interfaces and also N
      IMPLICIT NONE
      Real, Allocatable :: Z(:)
      Real :: X,Y
      Integer :: iF1,iF2
      CALL RANDOM_SEED()
      Call RANDOM_NUMBER(Y)
      X = 10**Y
      iF1 = F1(X,N)  !<-- N gets defined here
      Allocate(Z(N)) !<-- N gets used here
      iF2 = F2(Y,Z)  !<-- and here indirectly
      Write(*,*)'N = ',N
      Write(*,*)'iF1+iF2=',iF1+iF2
      Deallocate(Z)
      End Program TestDriver

C:\temp\user>dir
 Volume in drive C is OS
 Volume Serial Number is C4CA-BA12

 Directory of C:\temp\user

11/25/2021  06:58 PM    <DIR>          .
11/25/2021  06:58 PM    <DIR>          ..
11/25/2021  06:51 PM             9,728 bb.dll
11/25/2021  06:51 PM             1,742 bb.lib
11/25/2021  06:58 PM             1,257 p.f90
               3 File(s)         12,727 bytes
               2 Dir(s)  47,312,220,160 bytes free

C:\temp\user>ifort /c /standard-semantics p.f90
Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation.  All rights reserved.


C:\temp\user>link p.obj bb.lib /subsystem:console /out:p.exe
Microsoft (R) Incremental Linker Version 14.27.29112.0
Copyright (C) Microsoft Corporation.  All rights reserved.


C:\temp\user>p.exe
 N =  9
 iF1+iF2= 131

C:\temp\user>

So you can see here how the integration module helps you build a program to consume that blackbox DLL without issues.

Note with this, no changes whatsoever to the blackbox DLL are envisioned, only some knowledge its methods, 'F1' and 'F2', and their interfaces that you are now creating explicitly whilst employing bind(C). 

Note also what is shown above are the raw steps toward this from the Intel Fortran command prompt.  Once you understand them, you can setup your Visual Studio project to achieve the same.

[SunMesa]

@FortranFan 

That is a very clever (and evidently functional!) suggestion. I will code this up and test it in the morning, Thanksgiving activities have commandeered me for now... thanks for your insight-

[IanH]

---

@FortranFan wrote:

...

      module Ibb_m
      ! Module to integrate your code with a blackbox DLL

         integer :: N   !<-- Enclose N as a module entity
         INTERFACE
           Integer(C_INT) Function F1(X,N) Bind(C, name="F1") !<-- Note the name clause
           USE, INTRINSIC :: ISO_C_BINDING
           IMPLICIT NONE
           Real(C_FLOAT), Intent(IN) :: X
           Integer(C_INT), Intent(OUT) :: N
           End Function F1
         END INTERFACE
         INTERFACE
           Integer(C_INT) Function F2(Y,Z) Bind(C, name="F2") !<-- Note the name clause
           USE, INTRINSIC :: ISO_C_BINDING
           import :: N   !<-- Note this statement to import N from the host
           IMPLICIT NONE
           Real(C_FLOAT), INTENT(IN) :: Y
           Real(C_FLOAT), INTENT(OUT) :: Z(N)
           End Function F2
         END INTERFACE

      end module 

...
Note with this, no changes whatsoever to the blackbox DLL are envisioned, only some knowledge its methods, 'F1' and 'F2', and their interfaces that you are now creating explicitly whilst employing bind(C). 

How is the N module variable defined?  If the code for f1 is in a DLL, how does it know about a module variable in the exe?

(All these issues with the use of "N" in the interface could be avoided if the relevant dummy was an assumed size array.)

[SunMesa]

@FortranFan 

I'm happy to verify that your proposed integration solution works superbly! I confess I don't entirely understand how... the secret seems to lie in the "import :: N" statement added to the interface block for function F2 (which notably has no counterpart in the actual F2 function in the DLL).

Curiously, I was unable to build the package from the command line, as in your post (although I must note I have limited experience in doing so). I was able to compile all the pieces with ifort, but the linker failed with the message that the Ibb_m.obj was 'invalid or corrupt'.  However, in Visual Studio, the entire package built and executed flawlessly as-is, with no modifications to my default VS environment or options. I tried copying the verbatum link command from the Visual Studio build log into a windows command line (after stripping off some options that were flagged as unrecognizable), but got the same error complaining about Ibb_m.obj. I'll continue investigating.

In any case, this was a resounding success, and I have flagged your post as the solution. The translation to my real-world DLL and driver should be straightforward. Thanks again for your help!