Linking Lapack 95 in a DLL

[Frank_M]

 I cant seem to figure out how to set the Properties to compile and link MKL95 in a DLL.

Here is the code:

SUBROUTINE GET_ROOTS(phi, philen, side, output)
!DEC$ ATTRIBUTES DLLEXPORT::GET_ROOTS
    USE mkl95_precision
    USE mkl95_lapack, ONLY: HSEQR
    INTEGER, INTENT(IN) :: philen
    REAL(8), INTENT(IN) :: side
    REAL(8), DIMENSION(philen), INTENT(INOUT) :: phi
    COMPLEX(16), DIMENSION(philen-1), INTENT(INOUT) :: output
    COMPLEX(16), DIMENSION(:,:), ALLOCATABLE :: matrixA
    COMPLEX(16), DIMENSION(:), ALLOCATABLE :: roots
    INTEGER :: status, dim, i
    dim = philen-1
    ALLOCATE(matrixA(dim,dim), STAT = status)
    ALLOCATE(roots(dim), STAT = status)
    roots = 0.0_8
    matrixA = 0.0_8
     DO i = 1, dim-1
        matrixA(1,i) = -1.0_8 * phi(philen-i)/phi(philen)
        matrixA(i+1,i) = 1.0_8
    END DO 
    matrixA(1,dim) = -1.0_8 * phi(1)/phi(philen)      
    CALL HSEQR(matrixA,roots)
    output = roots
    DEALLOCATE(matrixA)
    DEALLOCATE(roots)
    return
END SUBROUTINE GET_ROOTS

Here are the compilers errors:

>GET_ROOTS.f90
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(8): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [MKL95_PRECISION]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(9): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [MKL95_LAPACK]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(27): error #6406: Conflicting attributes or multiple declaration of name.   [HSEQR]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(9): error #6580: Name in only-list does not exist.   [HSEQR]
1>compilation aborted for C:\Users\FrankM\documents\visual studio 2012\Projects\GET_ROOTS\GET_ROOTS\GET_ROOTS.f90 (code 1)

of course the first thing I tried was to set

Sequential (/Qmkl:sequential)

I 've also tried using the advisor and get the same messages.

Help.

[mecej4]

You did not state which version of MKL you are using, so the following may be a guess: you should probably have USE F95_PRECISION on line-3 and USE LAPACK95, ONLY: HSEQR in line-4. The correct module names may be inferred by looking at the names of the .mod files in the mkl/include/<arch> directory.

[TimP]

mkl_lapack95 (for X64, mkl_lapack95_lp64 or mkl_lapack95_ilp64) must be specified in the link line.  The USE and /Qmkl aren't sufficient.  If you want details, use the link advisor.  You do need to know whether you are building for ia32 or Intel64.

I'm not trying to understand what you mean by "in a dll."  A full command line or VS build.html might clarify this.

[Frank_M]

I'm using MKL 11.3.

The use of a DLL is for testing. Specifically what is the error in the roots of a characteristic equation.

The problem is that I don't know how to set the Property Pages. All the examples provided by Intel assume the use of a command line or a make file.The adviser spits out:

Use this link line:
 mkl_lapack95.lib mkl_intel_c_dll.lib mkl_core_dll.lib mkl_sequential_dll.lib

Compiler Options:
 /module:"%MKLROOT%"\include\ia32 -I"%MKLROOT%"\include

notes:
 Set the PATH, LIB and INCLUDE environment variables in the command shell using one of mklvars script files in the 'bin' subdirectory of the Intel(R) 
MKL installation directory. Please see also the Intel(R) MKL User Guide.
to Set the F95ROOT variable. Fortran 95 interfaces are compiler-dependent. 
The Fortran 95 interfaces and wrappers are delivered as sources, build the appropriate library and modules 
with your compiler. Please see also the Intel(R) MKL User Guide. 

Where and how do I set the Property Pages so I don't have to use a make file?

[Gennady_F_Intel]

you may look at the MKL User'sGuide - see chapter "Configuring the Microsoft Visual C/C++* Development System to Link with Intel® MKL" where you will find the detailed description how to properly set all items under VS.

[Frank_M]

Premier support was able to show me that lapack needed precision specified with a SP or DP.

Unfortunately, although the code will compile without kicking out any errors, I get the following error when actually running the code. 

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.getroots(Double[] , Int32& , Double[] , Double[] ).

The error disappears when the call is removed so it seems that mkl is still not getting linked. How do I statically link MKL to a dll?

I've spent hours and hours reading articles and manuals, tried dozens and dozens of combinations of compiler options, includes, and hard code paths to the mod files. Does anybody know the secret combination?

[mecej4]

Your problem may be traced to a possible misunderstanding of what COMPLEX(16) means. As you can see in https://software.intel.com/en-us/node/526047, specifying a kind number of 16 implies that you want quadruple precision, i.e., the same as COMPLEX*32 (the "32" stands for 32 bytes = 16 bytes each for the real and imaginary parts).

MKL routines do not support quadruple precision, and quadruple precision does not have direct hardware/microcode implementation in X86/X64 CPUs. Consequently, compilation of Fortran codes, such as the one in #1, will fail if interface checking is enabled. If interface checking is not done, linker and/or run-time errors will occur.

[Frank_M]

mecej4 wrote:

Your problem may be traced to a possible misunderstanding of what COMPLEX(16) means. As you can see in https://software.intel.com/en-us/node/526047, specifying a kind number of 16 implies that you want quadruple precision, i.e., the same as COMPLEX*32 (the "32" stands for 32 bytes = 16 bytes each for the real and imaginary parts).

MKL routines do not support quadruple precision, and quadruple precision does not have direct hardware/microcode implementation in X86/X64 CPUs. Consequently, compilation of Fortran codes, such as the one in #1, will fail if interface checking is enabled. If interface checking is not done, linker and/or run-time errors will occur.

Thanks but that is no longer the issue. The issue is how do you statically link a fortran dll to mkl? Premier support insists that the link line advisor provides a solution, but it doesn't. The LLA says to add the libraries " mkl_lapack95.lib mkl_intel_c.lib mkl_core.lib mkl_intel_thread.lib." Well I've added the libraries.

Compile"

1>GET_ROOTS - 0 error(s), 0 warning(s)
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

Test by trying to call the dll:

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.getroots(Double[] , Int32& , Double[] , Double[] ).

Remove the call to LApack, error goes away.

The only clues I found are in this thread:

https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/290594

Once again the issue is How do I link a fortran dll to MKL?

[Ying_H_Intel]

Hi Frank,

According to your error message, it seems there is issue when call the Fortran DLL in .Net.

And Vadimir mentioned, there may be issue when call fortran 95 Dll in .Net. Could you please provide us your test net program so we can test?

Other possiblity: As fortran call conversion is not different c dll, there are some special things one may consider when call fotran in other lauguage. You may refer to some sample code in Intel Fotran about mixed language call.  like

The routine name must be in uppercase unless the Fortran code

includes an appropriate ALIAS attribute.

*/

Best Regards,

Ying

[Frank_M]

Current state of code:

!  GET_ROOTS.f90     
!    
!  FUNCTIONS/SUBROUTINES exported from GET_ROOTS.dll:
!  GET_ROOTS - subroutine  
!
SUBROUTINE getroots(phi, philen,rootR, rootI)
!DEC$ ATTRIBUTES DLLEXPORT::getroots 
!DEC$ ATTRIBUTES ALIAS:'getroots' :: getroots
    USE f95_precision, ONLY: WP => DP
    USE lapack95, ONLY: HSEQR     

    INTEGER(4),     INTENT(IN)                              :: philen
    REAL(8),        DIMENSION(philen),      INTENT(INOUT)   :: phi
    REAL(8),        DIMENSION(philen-1),    INTENT(INOUT)   :: rootR
    REAL(8),        DIMENSION(philen-1),    INTENT(INOUT)   :: rootI
    COMPLEX(WP),    DIMENSION(:,:),         ALLOCATABLE     :: matrixA
    COMPLEX(WP),    DIMENSION(:),           ALLOCATABLE     :: roots
    INTEGER :: status, n, i
    n = philen-1
    ALLOCATE(matrixA(n,n), STAT = status)
    ALLOCATE(roots(n), STAT = status)
    roots = 0.0_8
    matrixA = 0.0_8
    DO i = 1, n-1
        matrixA(1,i) = -1.0_8 * phi(philen-i)/phi(philen)
        matrixA(i+1,i) = 1.0_8
    END DO 
    matrixA(1,n) = -1.0_8 * phi(1)/phi(philen)      
    CALL HSEQR(matrixA, roots)
    rootR = REAL(roots)
    rootI = AIMAG(roots)
    DEALLOCATE(matrixA)
    DEALLOCATE(roots)
    return
END SUBROUTINE getroots

Current state of test program:

Needs["NETLink`"]

ReinstallNET[
  "Force32Bit" -> True];(*or InstallNET["Force32Bit"\[Rule]True]*)

libName = "C:\\Users\\FrankM\\Documents\\Visual Studio 2012\\Projects\
\\GET_ROOTS\\GET_ROOTS\\Debug\\GET_ROOTS.dll"

"C:\\Users\\FrankM\\Documents\\Visual Studio \
2012\\Projects\\GET_ROOTS\\GET_ROOTS\\Debug\\GET_ROOTS.dll"

FileExistsQ[libName]

True

TestSubroutine = 
 DefineDLLFunction["getroots", libName, 
  "void", {"Double[]", "int*", "Double[]", "Double[]"}]

Function[Null, 
 If[NETLink`DLL`Private`checkArgCount["getroots", {##1}, 4], 
  Wolfram`NETLink`DynamicDLLNamespace`DLLWrapper1`getroots[##1], \
$Failed], {HoldAll}]

phi = RandomReal[{-100, 100}, 10];
philen = Length[phi];
rootR = MakeNETObject[Evaluate[Table[0.0, {philen - 1}]]];
rootI = MakeNETObject[Evaluate[Table[0.0, {philen - 1}]]]; 


TestSubroutine[phi, philen, rootR, rootI]

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.getroots(Double[] , Int32& , Double[] , Double[] ).

$Failed





out1 = NETObjectToExpression[rootR]

{0., 0., 0., 0., 0., 0., 0., 0., 0.}

out2 = NETObjectToExpression[rootI]

{0., 0., 0., 0., 0., 0., 0., 0., 0.}

The test program is a mathemataca notebook.

[Frank_M]

Here is the current state of everything.

[Ying_H_Intel]

Hi Frank,

I'm not familiar with mathematica environment  and it's calling conversion rule, but  share my view about your issue.

1) according to the error  An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

(Double[] , Int32& , Double[] , Double[] )

TestSubroutine[phi, philen, rootR, rootI]

How about TestSubroutine[phi, &philen, rootR, rootI] ? 

2) About the dll GET_ROOTS.dll's dependency.  ( you can try the depencency walker to check). 

As i see, the GET_ROOTS.dll seems still link with MKL dynamic library (mkl_core.dll etc). if with static library , the thing may simple, for example, test program will not requir the mkl dll again. 

You can change your project to static library easily

1) switch off  Fortran=>library=> Use Intel MKL library = NO. 

2) Add the library path as i mentioned in that article manually. 

Best Regards,
Ying 

[Frank_M]

Mathematica uses netlink to talk to windows. It is similar to C#.

here is a link if your interested: http://reference.wolfram.com/language/NETLink/tutorial/CallingNETFromTheWolframLanguage.html#24546

TestSubroutine[phi, &philen, rootR, rootI]  returns a "Syntax::sntxf: "TestSubroutine[phi," cannot be followed by "&philen,rootR,rootI]".

Without the     CALL HSEQR(matrixA, roots) Mathematica calls and runs the dll. When I create rootR it is asigned 0.0.

So to test the test routine I comment out Call HSEQR and set rootR to 1.0 in the dll.

In[11]:= out1 = NETObjectToExpression[rootR]

Out[11]= {1., 1., 1., 1., 1., 1., 1., 1., 1.}

I'll start on suggestion 2.

[mecej4]

You can write a small driver program in Fortran or C to call your DLL routines with the same arguments as you expect Mathematica to pass to the DLL. We are probably going to be more helpful with such a driver than with a complex program such as Mathematica.

After fixing any errors in the DLL, you can then see to it that the linkage between Mathematica and the DLL is bug-free.

[Frank_M]

Hi Ying.

The method you showed me to static link the dll to mkl produced the same error. It even produced the same error when I commented out the Call HSEQR. I even tried just replacing my solution with yours and changing the test notebook to account for the different parameter list.

So perhaps Qmkl is better and the link to mkl is not he problem. I expect the solution is what elengie found. I'll attach the output from the dependency walker.

@ mecej4

 I'll start on a fortran test program.

[mecej4]

Frank_M, I think that you would be more successful if you built up the application in steps.

The source code of the DLL that I used with the driver below is the same as what you gave in #10, but without any directives. I built the DLL with the command:

ifort /LD /Qmkl roots.f90 mkl_lapack95.lib /link /export:GETROOTS

I tested using a Fortran driver tst.f90:

program xroots
integer :: philen = 10
double precision :: phi(10) =  &
   [10.7079, 31.3279, -5.48742, -18.5684, -28.8292, &
   -96.5144, -76.2258, 8.62183, 21.7482, 54.6085]
double precision, dimension(9) :: rootR,rootI
!
call getroots(phi,philen,rootR,rootI)
write(*,'(i3,2ES13.4)')(i,rootR(i),rootI(i),i=1,9)
end program

Compiling the driver with the command

ifort tst.f90 roots.lib

and running the EXE gave the results:

  1   1.2983E+00   5.5511E-16
  2  -3.9886E-01   1.1530E+00
  3  -3.9886E-01  -1.1530E+00
  4   1.6903E-01   7.7653E-01
  5   6.7501E-01  -2.6411E-16
  6   1.6903E-01  -7.7653E-01
  7  -7.8450E-01   2.8051E-01
  8  -7.8450E-01  -2.8051E-01
  9  -3.4289E-01   0.0000E+00

If this is what you wanted, you can now try calling a similar DLL from Mathematica, after making adjustments for argument passing conventions and linkage rules.

[Frank_M]

I am not able to get the tst.f90 to link. Where does tst.90 need to be in relation to GET_ROOTS.dll  and the GET_ROOTS.lib? Should everything be in the same folder? (I tried having all in same folder and received fatal error LINK120: i unresolved externals

I compiled the GET_ROOTS.dll using  ifort /LD /Qmkl GET_ROOTS.f90 mkl_lapack95.lib /link /export:GET_ROOTS

and it compiled fine (after some name changing). When trying to call from mathematica I received a 

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.GET_ROOTS(Double[] , Int32& , Double[] , Double[] ).

[Ying_H_Intel]

Happy New year to all of you! 

from the dependency walk Roots dep.png ,  the problem seems still here. 

1.   dynamic library vs.static library 

the  mkl_core.dll, mkl_seqential.dll are dynamic libraries. it was needed in search path when run your application. 

2. dependency library. 

The dynamic library will depend on other cpu-specific library, like libmkl_avx.dll etc.  So when you run your appplication,  all of dependency library are needed in search path.

So if the get_root.dll has such dependency, you may need to make all related dll in the mathimatic search path.    for example, you may try copy all mkl dlls ( I mean the whole dlls file in mkl redist directory to your exe folder or windows system32 folder etc). 

On the other hand,  that is why I ask you try static link instead of dll link :   manually add  in property page => Linker => input => additional dependency: mkl_lapack95.lib mkl_intel_c.lib mkl_sequential.lib mkl_core.lib

not Qmkl:sequential (it will link dynamic library). 

I build one  one C code to call the library as mecej4 (thank a lot) to show this.   I upload them later. 

Best Regards,

Ying 

[Ying_H_Intel]

Upload the C call fotran95 test case and screencopy doc. 

Best Regards,

Ying