topic Linking Lapack 95 in a DLL in Intel® oneAPI Math Kernel Library

Linking Lapack 95 in a DLL

Frank_M — Mon, 30 Nov 2015 13:05:39 GMT

I cant seem to figure out how to set the Properties to compile and link MKL95 in a DLL.

Here is the code:

SUBROUTINE GET_ROOTS(phi, philen, side, output)
!DEC$ ATTRIBUTES DLLEXPORT::GET_ROOTS
    USE mkl95_precision
    USE mkl95_lapack, ONLY: HSEQR
    INTEGER, INTENT(IN) :: philen
    REAL(8), INTENT(IN) :: side
    REAL(8), DIMENSION(philen), INTENT(INOUT) :: phi
    COMPLEX(16), DIMENSION(philen-1), INTENT(INOUT) :: output
    COMPLEX(16), DIMENSION(:,:), ALLOCATABLE :: matrixA
    COMPLEX(16), DIMENSION(:), ALLOCATABLE :: roots
    INTEGER :: status, dim, i
    dim = philen-1
    ALLOCATE(matrixA(dim,dim), STAT = status)
    ALLOCATE(roots(dim), STAT = status)
    roots = 0.0_8
    matrixA = 0.0_8
     DO i = 1, dim-1
        matrixA(1,i) = -1.0_8 * phi(philen-i)/phi(philen)
        matrixA(i+1,i) = 1.0_8
    END DO 
    matrixA(1,dim) = -1.0_8 * phi(1)/phi(philen)      
    CALL HSEQR(matrixA,roots)
    output = roots
    DEALLOCATE(matrixA)
    DEALLOCATE(roots)
    return
END SUBROUTINE GET_ROOTS

Here are the compilers errors:

>GET_ROOTS.f90
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(8): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [MKL95_PRECISION]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(9): error #7002: Error in opening the compiled module file.  Check INCLUDE paths.   [MKL95_LAPACK]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(27): error #6406: Conflicting attributes or multiple declaration of name.   [HSEQR]
GET_ROOTS\GET_ROOTS\GET_ROOTS.f90(9): error #6580: Name in only-list does not exist.   [HSEQR]
1>compilation aborted for C:\Users\FrankM\documents\visual studio 2012\Projects\GET_ROOTS\GET_ROOTS\GET_ROOTS.f90 (code 1)

of course the first thing I tried was to set

Sequential (/Qmkl:sequential)

I 've also tried using the advisor and get the same messages.

Help.

You did not state which

mecej4 — Mon, 30 Nov 2015 13:15:10 GMT

You did not state which version of MKL you are using, so the following may be a guess: you should probably have USE F95_PRECISION on line-3 and USE LAPACK95, ONLY: HSEQR in line-4. The correct module names may be inferred by looking at the names of the .mod files in the mkl/include/<arch> directory.

mkl_lapack95 (for X64, mkl

TimP — Mon, 30 Nov 2015 16:40:00 GMT

mkl_lapack95 (for X64, mkl_lapack95_lp64 or mkl_lapack95_ilp64) must be specified in the link line. The USE and /Qmkl aren't sufficient. If you want details, use the link advisor. You do need to know whether you are building for ia32 or Intel64.

I'm not trying to understand what you mean by "in a dll." A full command line or VS build.html might clarify this.

I'm using MKL 11.3.

Frank_M — Wed, 02 Dec 2015 14:13:04 GMT

I'm using MKL 11.3.

The use of a DLL is for testing. Specifically what is the error in the roots of a characteristic equation.

The problem is that I don't know how to set the Property Pages. All the examples provided by Intel assume the use of a command line or a make file.The adviser spits out:

Use this link line:
 mkl_lapack95.lib mkl_intel_c_dll.lib mkl_core_dll.lib mkl_sequential_dll.lib

Compiler Options:
 /module:"%MKLROOT%"\include\ia32 -I"%MKLROOT%"\include

notes:
 Set the PATH, LIB and INCLUDE environment variables in the command shell using one of mklvars script files in the 'bin' subdirectory of the Intel(R) 
MKL installation directory. Please see also the Intel(R) MKL User Guide.
to Set the F95ROOT variable. Fortran 95 interfaces are compiler-dependent. 
The Fortran 95 interfaces and wrappers are delivered as sources, build the appropriate library and modules 
with your compiler. Please see also the Intel(R) MKL User Guide.

Where and how do I set the Property Pages so I don't have to use a make file?

you may look at the MKL User

Gennady_F_Intel — Wed, 02 Dec 2015 15:43:40 GMT

you may look at the MKL User'sGuide - see chapter "Configuring the Microsoft Visual C/C++* Development System to Link with Intel® MKL" where you will find the detailed description how to properly set all items under VS.

Quote:Gennady Fedorov (Intel)

Frank_M — Thu, 24 Dec 2015 11:50:00 GMT

Premier support was able to show me that lapack needed precision specified with a SP or DP.

Unfortunately, although the code will compile without kicking out any errors, I get the following error when actually running the code.

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.getroots(Double[] , Int32& , Double[] , Double[] ).

The error disappears when the call is removed so it seems that mkl is still not getting linked. How do I statically link MKL to a dll?

I've spent hours and hours reading articles and manuals, tried dozens and dozens of combinations of compiler options, includes, and hard code paths to the mod files. Does anybody know the secret combination?

Your problem may be traced to

mecej4 — Thu, 24 Dec 2015 13:34:00 GMT

Your problem may be traced to a possible misunderstanding of what COMPLEX(16) means. As you can see in https://software.intel.com/en-us/node/526047, specifying a kind number of 16 implies that you want quadruple precision, i.e., the same as COMPLEX*32 (the "32" stands for 32 bytes = 16 bytes each for the real and imaginary parts).

MKL routines do not support quadruple precision, and quadruple precision does not have direct hardware/microcode implementation in X86/X64 CPUs. Consequently, compilation of Fortran codes, such as the one in #1, will fail if interface checking is enabled. If interface checking is not done, linker and/or run-time errors will occur.

Quote:mecej4 wrote:

Frank_M — Thu, 24 Dec 2015 22:43:51 GMT

mecej4 wrote:

Your problem may be traced to a possible misunderstanding of what COMPLEX(16) means. As you can see in https://software.intel.com/en-us/node/526047, specifying a kind number of 16 implies that you want quadruple precision, i.e., the same as COMPLEX*32 (the "32" stands for 32 bytes = 16 bytes each for the real and imaginary parts).

MKL routines do not support quadruple precision, and quadruple precision does not have direct hardware/microcode implementation in X86/X64 CPUs. Consequently, compilation of Fortran codes, such as the one in #1, will fail if interface checking is enabled. If interface checking is not done, linker and/or run-time errors will occur.

Thanks but that is no longer the issue. The issue is how do you statically link a fortran dll to mkl? Premier support insists that the link line advisor provides a solution, but it doesn't. The LLA says to add the libraries " mkl_lapack95.lib mkl_intel_c.lib mkl_core.lib mkl_intel_thread.lib." Well I've added the libraries.

Compile"

1>GET_ROOTS - 0 error(s), 0 warning(s)
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========

Test by trying to call the dll:

Remove the call to LApack, error goes away.

The only clues I found are in this thread:

https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/290594

Once again the issue is How do I link a fortran dll to MKL?

Hi Frank,

Ying_H_Intel — Fri, 25 Dec 2015 01:32:37 GMT

Hi Frank,

According to your error message, it seems there is issue when call the Fortran DLL in .Net.

And Vadimir mentioned, there may be issue when call fortran 95 Dll in .Net. Could you please provide us your test net program so we can test?

Other possiblity: As fortran call conversion is not different c dll, there are some special things one may consider when call fotran in other lauguage. You may refer to some sample code in Intel Fotran about mixed language call. like

The routine name must be in uppercase unless the Fortran code

includes an appropriate ALIAS attribute.

Best Regards,

Ying

Current state of code:

Frank_M — Fri, 25 Dec 2015 03:54:17 GMT

Current state of code:

!  GET_ROOTS.f90     
!    
!  FUNCTIONS/SUBROUTINES exported from GET_ROOTS.dll:
!  GET_ROOTS - subroutine  
!
SUBROUTINE getroots(phi, philen,rootR, rootI)
!DEC$ ATTRIBUTES DLLEXPORT::getroots 
!DEC$ ATTRIBUTES ALIAS:'getroots' :: getroots
    USE f95_precision, ONLY: WP => DP
    USE lapack95, ONLY: HSEQR     

    INTEGER(4),     INTENT(IN)                              :: philen
    REAL(8),        DIMENSION(philen),      INTENT(INOUT)   :: phi
    REAL(8),        DIMENSION(philen-1),    INTENT(INOUT)   :: rootR
    REAL(8),        DIMENSION(philen-1),    INTENT(INOUT)   :: rootI
    COMPLEX(WP),    DIMENSION(:,:),         ALLOCATABLE     :: matrixA
    COMPLEX(WP),    DIMENSION(:),           ALLOCATABLE     :: roots
    INTEGER :: status, n, i
    n = philen-1
    ALLOCATE(matrixA(n,n), STAT = status)
    ALLOCATE(roots(n), STAT = status)
    roots = 0.0_8
    matrixA = 0.0_8
    DO i = 1, n-1
        matrixA(1,i) = -1.0_8 * phi(philen-i)/phi(philen)
        matrixA(i+1,i) = 1.0_8
    END DO 
    matrixA(1,n) = -1.0_8 * phi(1)/phi(philen)      
    CALL HSEQR(matrixA, roots)
    rootR = REAL(roots)
    rootI = AIMAG(roots)
    DEALLOCATE(matrixA)
    DEALLOCATE(roots)
    return
END SUBROUTINE getroots

Current state of test program:

Needs["NETLink`"]

ReinstallNET[
  "Force32Bit" -> True];(*or InstallNET["Force32Bit"\[Rule]True]*)

libName = "C:\\Users\\FrankM\\Documents\\Visual Studio 2012\\Projects\
\\GET_ROOTS\\GET_ROOTS\\Debug\\GET_ROOTS.dll"

"C:\\Users\\FrankM\\Documents\\Visual Studio \
2012\\Projects\\GET_ROOTS\\GET_ROOTS\\Debug\\GET_ROOTS.dll"

FileExistsQ[libName]

True

TestSubroutine = 
 DefineDLLFunction["getroots", libName, 
  "void", {"Double[]", "int*", "Double[]", "Double[]"}]

Function[Null, 
 If[NETLink`DLL`Private`checkArgCount["getroots", {##1}, 4], 
  Wolfram`NETLink`DynamicDLLNamespace`DLLWrapper1`getroots[##1], \
$Failed], {HoldAll}]

phi = RandomReal[{-100, 100}, 10];
philen = Length[phi];
rootR = MakeNETObject[Evaluate[Table[0.0, {philen - 1}]]];
rootI = MakeNETObject[Evaluate[Table[0.0, {philen - 1}]]]; 


TestSubroutine[phi, philen, rootR, rootI]

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
   at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.getroots(Double[] , Int32& , Double[] , Double[] ).

$Failed





out1 = NETObjectToExpression[rootR]

{0., 0., 0., 0., 0., 0., 0., 0., 0.}

out2 = NETObjectToExpression[rootI]

{0., 0., 0., 0., 0., 0., 0., 0., 0.}

The test program is a mathemataca notebook.

Here is the current state of

Frank_M — Fri, 25 Dec 2015 04:03:34 GMT

Here is the current state of everything.

Hi Frank,

Ying_H_Intel — Fri, 25 Dec 2015 07:24:47 GMT

Hi Frank,

I'm not familiar with mathematica environment and it's calling conversion rule, but share my view about your issue.

1) according to the error An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)

(Double[] , Int32& , Double[] , Double[] )

TestSubroutine[phi, philen, rootR, rootI]

How about TestSubroutine[phi, &philen, rootR, rootI] ?

2) About the dll GET_ROOTS.dll's dependency. ( you can try the depencency walker to check).

As i see, the GET_ROOTS.dll seems still link with MKL dynamic library (mkl_core.dll etc). if with static library , the thing may simple, for example, test program will not requir the mkl dll again.

You can change your project to static library easily

1) switch off Fortran=>library=> Use Intel MKL library = NO.

2) Add the library path as i mentioned in that article manually.

Best Regards,
Ying

Mathematica uses netlink to

Frank_M — Fri, 25 Dec 2015 08:02:56 GMT

Mathematica uses netlink to talk to windows. It is similar to C#.

here is a link if your interested: http://reference.wolfram.com/language/NETLink/tutorial/CallingNETFromTheWolframLanguage.html#24546

TestSubroutine[phi, &philen, rootR, rootI] returns a "Syntax::sntxf: "TestSubroutine[phi," cannot be followed by "&philen,rootR,rootI]".

Without the CALL HSEQR(matrixA, roots) Mathematica calls and runs the dll. When I create rootR it is asigned 0.0.

So to test the test routine I comment out Call HSEQR and set rootR to 1.0 in the dll.

In[11]:= out1 = NETObjectToExpression[rootR]

Out[11]= {1., 1., 1., 1., 1., 1., 1., 1., 1.}

I'll start on suggestion 2.

You can write a small driver

mecej4 — Fri, 25 Dec 2015 11:01:28 GMT

You can write a small driver program in Fortran or C to call your DLL routines with the same arguments as you expect Mathematica to pass to the DLL. We are probably going to be more helpful with such a driver than with a complex program such as Mathematica.

After fixing any errors in the DLL, you can then see to it that the linkage between Mathematica and the DLL is bug-free.

Hi Ying.

Frank_M — Sat, 26 Dec 2015 12:17:34 GMT

Hi Ying.

The method you showed me to static link the dll to mkl produced the same error. It even produced the same error when I commented out the Call HSEQR. I even tried just replacing my solution with yours and changing the test notebook to account for the different parameter list.

So perhaps Qmkl is better and the link to mkl is not he problem. I expect the solution is what elengie found. I'll attach the output from the dependency walker.

@ mecej4

I'll start on a fortran test program.

Frank_M, I think that you

mecej4 — Sat, 26 Dec 2015 13:13:00 GMT

Frank_M, I think that you would be more successful if you built up the application in steps.

The source code of the DLL that I used with the driver below is the same as what you gave in #10, but without any directives. I built the DLL with the command:

ifort /LD /Qmkl roots.f90 mkl_lapack95.lib /link /export:GETROOTS

I tested using a Fortran driver tst.f90:

program xroots
integer :: philen = 10
double precision :: phi(10) =  &
   [10.7079, 31.3279, -5.48742, -18.5684, -28.8292, &
   -96.5144, -76.2258, 8.62183, 21.7482, 54.6085]
double precision, dimension(9) :: rootR,rootI
!
call getroots(phi,philen,rootR,rootI)
write(*,'(i3,2ES13.4)')(i,rootR(i),rootI(i),i=1,9)
end program

Compiling the driver with the command

ifort tst.f90 roots.lib

and running the EXE gave the results:

  1   1.2983E+00   5.5511E-16
  2  -3.9886E-01   1.1530E+00
  3  -3.9886E-01  -1.1530E+00
  4   1.6903E-01   7.7653E-01
  5   6.7501E-01  -2.6411E-16
  6   1.6903E-01  -7.7653E-01
  7  -7.8450E-01   2.8051E-01
  8  -7.8450E-01  -2.8051E-01
  9  -3.4289E-01   0.0000E+00

If this is what you wanted, you can now try calling a similar DLL from Mathematica, after making adjustments for argument passing conventions and linkage rules.

I am not able to get the tst

Frank_M — Sun, 03 Jan 2016 19:39:56 GMT

I am not able to get the tst.f90 to link. Where does tst.90 need to be in relation to GET_ROOTS.dll and the GET_ROOTS.lib? Should everything be in the same folder? (I tried having all in same folder and received fatal error LINK120: i unresolved externals

I compiled the GET_ROOTS.dll using ifort /LD /Qmkl GET_ROOTS.f90 mkl_lapack95.lib /link /export:GET_ROOTS

and it compiled fine (after some name changing). When trying to call from mathematica I received a

NET::netexcptn: A .NET exception occurred: System.BadImageFormatException: An attempt was made to load a program with an incorrect format. (Exception from HRESULT: 0x8007000B)
at Wolfram.NETLink.DynamicDLLNamespace.DLLWrapper1.GET_ROOTS(Double[] , Int32& , Double[] , Double[] ).

Happy New year to all of you!

Ying_H_Intel — Mon, 04 Jan 2016 01:32:45 GMT

Happy New year to all of you!

from the dependency walk Roots dep.png , the problem seems still here.

1. dynamic library vs.static library

the mkl_core.dll, mkl_seqential.dll are dynamic libraries. it was needed in search path when run your application.

2. dependency library.

The dynamic library will depend on other cpu-specific library, like libmkl_avx.dll etc. So when you run your appplication, all of dependency library are needed in search path.

So if the get_root.dll has such dependency, you may need to make all related dll in the mathimatic search path. for example, you may try copy all mkl dlls ( I mean the whole dlls file in mkl redist directory to your exe folder or windows system32 folder etc).

On the other hand, that is why I ask you try static link instead of dll link : manually add in property page => Linker => input => additional dependency: mkl_lapack95.lib mkl_intel_c.lib mkl_sequential.lib mkl_core.lib

not Qmkl:sequential (it will link dynamic library).

I build one one C code to call the library as mecej4 (thank a lot) to show this. I upload them later.

Best Regards,

Ying

Upload the C call fotran95

Ying_H_Intel — Mon, 04 Jan 2016 02:04:18 GMT

Upload the C call fotran95 test case and screencopy doc.

Best Regards,

Ying