using CLAPACK in mkl

[itabhiyanta]

Hi

I recently installed shifted my code to the intel compiler.
I have been using the clapack routines as provided in the ATLAS BLAS library and i am also using clapack for BLAS operations like ddot, daxpy, dnrm2 etc.

My code does not seem to compile correctly. i get undefined references fro all my clapack calls
clapack_dpotrs, clapack_dpotrf, clapack_daxpy clapack_ddot clapack_dnrm2. all of them are flagged as undefined references

I am including mkl.h and mkl_lapack.h (i couldn't find mkl_lapacke.h as suggested in the documentation)

the library component of my linkline looks like the following. whereINLIB2 is the path
opt/intel/Compiler/11.1/073/mkl/lib/em64t/

-Wl,--start-group -L$(INLIB2)/mkl_c.lib -L$(INLIB)/libmkl_intel_ilp64.a -L$(INLIB2)/libmkl_sequential.a -L$(INLIB2)/libmkl_core.a -Wl,--end-group -lpthread

I was earlier using ATLAS. My code development environment is for linux 64 bit environment. I am running the code on a Xeon machine.

I would really appreciate if someone could guide me where am i going wrong. I must mention the reason i am using clapack is that i cam coding in C and i wish to stay with the row major convention of storing the matrices.

thanks for reading my query

rohit

[mecej4]

The C_Lapack interfaces were not delivered with the 11.1 compilers. You can use the Fortran 77 interfaces, or download the C_Lapack source form Netlib and build the interface library.

A better alternative is the LapackE interface, which you can download and build likewise; one advantage of LapackE over C_Lapack is that the former is now integrated into MKL in version 12.0 of the compiler (10.3 is the MKL version).

[itabhiyanta]

Hi mecej4

thanks for your quick reply. so if i download the new version of the compiler (under the non-commercial section of intel's site) and the MKL and build both of them and then use the same link line as i had pasted in my original question it should ideally work right?

the other solution you suggest would mean one more library (which i compile against icc) which results from the compilation of the clapack source at netlib right?

thanks

rohit

[itabhiyanta]

Hi


actually my cblas interface is also not working

host_deflakernel.c_o: In function `calculate_Ptx':
host_deflakernel.c:(.text+0x11a2): undefined reference to `clapack_dpotrs'
wrapper.c_o: In function `hostcall':
wrapper.c:(.text+0x5bf): undefined reference to `cblas_dnrm2'
wrapper.c:(.text+0x627): undefined reference to `cblas_dnrm2'
slave_DPCG.c_o: In function `slaveDPCG':
slave_DPCG.c:(.text+0x7a8): undefined reference to `cblas_dcopy'
slave_DPCG.c:(.text+0x7d0): undefined reference to `cblas_daxpy'
slave_DPCG.c:(.text+0x7f0): undefined reference to `cblas_dcopy'
slave_DPCG.c:(.text+0x815): undefined reference to `cblas_dcopy'
slave_DPCG.c:(.text+0x9e1): undefined reference to `cblas_daxpy'
slave_DPCG.c:(.text+0xe4a): undefined reference to `cblas_dcopy'
host_cgkernel.c_o: In function `seq_cgsolver':
host_cgkernel.c:(.text+0xaf8): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0xb1e): undefined reference to `cblas_daxpy'
host_cgkernel.c:(.text+0xb3c): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0xb5a): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0xc70): undefined reference to `CLAPACK_dpotrf'
host_cgkernel.c:(.text+0xca1): undefined reference to `CLAPACK_dpotrs'
host_cgkernel.c:(.text+0xdcd): undefined reference to `cblas_daxpy'
host_cgkernel.c:(.text+0x10d3): undefined reference to `cblas_dnrm2'
host_cgkernel.c:(.text+0x19eb): undefined reference to `clapack_dpotrs'
host_cgkernel.c_o: In function `solverhost':
host_cgkernel.c:(.text+0x3888): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0x38ae): undefined reference to `cblas_daxpy'
host_cgkernel.c:(.text+0x38cc): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0x38ea): undefined reference to `cblas_dcopy'
host_cgkernel.c:(.text+0x3a00): undefined reference to `CLAPACK_dpotrf'
host_cgkernel.c:(.text+0x3a31): undefined reference to `CLAPACK_dpotrs'
host_cgkernel.c:(.text+0x3b5d): undefined reference to `cblas_daxpy'
host_cgkernel.c:(.text+0x3e63): undefined reference to `cblas_dnrm2'
host_cgkernel.c:(.text+0x477b): undefined reference to `clapack_dpotrs'


i ma using compiler version 11.1 and mkl version is 10.3 on a linux system (64 bit).

i am using the following link line library setting
-L/opt/intel/Compiler/11.1/073/mkl/lib/em64t//libmkl_blas95_ilp64.a -L/opt/intel/Compiler/11.1/073/mkl/lib/em64t//libmkl_lapack95_ilp64.a -L/libmkl_intel_ilp64.a -L/opt/intel/Compiler/11.1/073/mkl/lib/em64t/libmkl_sequential.a -L/opt/intel/Compiler/11.1/073/mkl/lib/em64t//libmkl_core.a -lpthread -lm

please suggest/advise.

rohit

[TimP]

Since you are confused about the meaning of -L library path designations for linking, please compare with the recommendations of the link advisor posted at the top of this forum. You don't need -lpthread to support mkl_sequential. Remember that your use of ilp64 libraries corresponds with long int function parameters and corresponding headers (the headers with int data types won't be compatible).

[TimP]

Basically, you've inserted -L directives where they aren't wanted. Where it's used, -L has to point to a directory, to be used in combination with -lsomelib, which (in the case of MKL) would expand to libsomelib.so, so it could be used for dynamic linking.

[mecej4]

You undertook some steps between #2 and #3 without taking us into your confidence. I can guess what happened, however:

1. You did not use the Link Advisor properly, as TimP already pointed out. You must read the documentation and understand the -L and -l options of compilers (which pass these options to the linker, ld ).

2. You are including irrelevant libraries such as libmkl_lapack95...

3. If you built, say, libclapack.a, you needed to add the option -lclapack to the recommended link line from the MKL Link Advisor, since the Advisor does not know about the need for third party libraries.

[itabhiyanta]

hi mecej4

indeed i didn't understand it correctly. i corrected that however i still have a issue. let me start by telling what are the steps i took.

SO i downloaded clapack
passed to it the path of the BLAS library from intel and compiled using icc.
/opt/intel/Compiler/11.1/073/mkl/lib/em64t/libmkl_blas95_ilp64.a

that generated a lapack_LINUX.a

i added this to the corrected link line from the link advisor as follows

-L/opt/intel/Compiler/11.1/073/mkl/lib/intel64 -Wl,--start-group -lmkl_intel_ilp64 -lmkl_sequential -lmkl_core -Wl,--end-group ../CLAPACK-3.2.1/lapack_LINUX.a -lm -L.

however when i run the code which calls dpotrf i get the following error
mpi_CPU_all:28458 terminated with signal 11 at PC=2aaabab89516 SP=7fff7240e2d0. Backtrace:
/opt/intel/Compiler/11.1/073/mkl/lib/em64t/libmkl_mc.so(mkl_lapack_ps_dpotrf_u_small+0x246)[0x2aaabab89516]


my call looks like this
dpotrf_( &upper, &numdomains, E, &numdomains, &info);

upper is a charatcer = 'U', info is an integer, numdomains is an integer and E is a array of doubles.

am i still doing something wrong in compilation? if not then why is the mkl_lapack_ps_dpotrf failing?

rohit

[mecej4]

use> /opt/intel/Compiler/11.1/073/mkl/lib/em64t/libmkl_blas95_ilp64.a

This library is relevant only if you are using Fortran 9+. For C/C++programs calling MKL, it is completely useless.

> dpotrf_( &upper, &numdomains, E, &numdomains, &info); ... and E is a array of doubles

You built C-lapack, and specified it in the link line, but your code still calls only the Fortran 77 routines. You must try harder to understand what you are doing.

E is supposed to be a 2-D array, in column-major order. You have not shown how E was declared, allocated and initialized.

You must read the MKL documentation and understand how to pass matrix arguments from C to Fortran. Failure to do so will cause all sorts of mysterious run-time errors such as seg-faults, array overruns, using uninitialized data, etc.

Alternatively, you could use the C-lapack or Lapack-E interfaces. Doing either will still involve reading the corresponding documentation and issuing correct calls.

[itabhiyanta]

Hi mecej4

I have tried to read through the documentation. For my version of the compiler 11.1 and nkl 10.3. when i look at the Intel MKL function for the dpotrs and dpotrf function they mention to include
mkl_lapacke.h file. I do not have that in my installation (rather the servers' installation).So i had downloaded CLAPACK since earlier i was working with ATLAS and then the integration of clapack_dpotrs and clapack_dpotrf was without any such problems or hassle.

I understand that i might have initially given you the impression that i do not even know how to link against libraries. That was due to the way i wrote here on the forum and i can apologize for that again.

I need your help to fix this issue so i state it again.

ON A BROAD LEVEL
I am trying to use CLAPACK becuase i have C code. I want to now use Intel C compiler with Intel MKL and want to stay with the row-major format of calling routines.

ON THE FUNCTION LEVEL
I use cblas_dcopy, cblas_ddot etc. also from the clapack interface
The matrix E that i pass to dpotrf is a symetric square matrix stored in row major order. I first call dpotrf and then call dpotrs to get a result.

Later on i use the result that dpotrf produces in matrix E to call dpotrs repeatedly.
--------------------
I want to know

what files to include other than mkl.h to get the definition of this clapack routines which accept C interface. My understanding from the README at the netlib clapack project says it is dpotrf_ with arguments passed by reference (http://www.netlib.org/clapack/readme)

What i did.

I used the following link line (i am using MPI also so there are four slaves and 1 master for example and dpotrf and rs are called by the master)
LIB :=-L$(MKLROOT)/lib/intel64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -Wl,--end-group -openmp -lpthread -L$(CLAPCK_PATH)/my_clapack_library

You had suggested that i should provide the clapack libarary in this link line so i used that too. However like i had mentioned in the previous mail for a problem size of 1 million unknowns (i am writing out the code for conjugate gradient) i see this error about the segmentation fault.

I understand that the wrapper library just uses the BLAS routines from the Intel MKL and then one can use the routines from the CLAPACK implmentation and pass them arguments in C-style(row-major).

I request you to point out where all have i gone wrong.

thanks and regards

rohit

[mecej4]

Most of the advice given in this thread has gone unheeded, so let's try a different approach. Here is a small example of using the F77 interface to MKL from a C caller. There is no need to use CLapack or any library that is not already provided with Intel C 11.1 and MKL 10.2.

[cpp]#include 
#include 
#include 

int main(int argc,char *argv[]){
double A[3*3]={  2.0, -1.0,  0.0,
                 0.0,  2.0, -1.0,
                 0.0,  0.0,  2.0},
         b[3]={1.0,2.0,3.0};

int n=3,info,nrhs=1; char uplo='L';

dpotrf_(&uplo,&n,A,&n,&info);
dpotrs_(&uplo,&n,&nrhs,A,&n,b,&n,&info);
printf("Solution: %10.4f %10.4f %10.4fn",b[0],b[1],b[2]);
}
[/cpp]

Since only the lower triangle of A is used, I have used zeroes to initialize the upper triangle.

Compile and run:

[bash]$ icc -mkl dpotrx.c
$ ./a.out
Solution:     2.5000     4.0000     3.5000
[/bash]