topic Linux ...calling MKL 10.2 from Python in Intel® oneAPI Math Kernel Library

Linux ...calling MKL 10.2 from Python

AndrewC — Wed, 20 Oct 2010 16:29:32 GMT

I have a Python extension ( SWIG generated) that calls MKL. There were initially various problems with missing shared libraries at runtime when MKL dynamically loaded processor specific libraries, but that was resolved with --start-group, --end-group being passed to the linker.

For example

MKLLIBS=-L$(MKLROOT)/lib/em64t $(MKLROOT)/lib/em64t/libmkl_solver_lp64.a -Wl,--start-group -lmkl_intel_lp64 -lmkl_intel_thread -lmkl_core -lmkl_mc -lmkl_p4n -lmkl_mc3 -lmkl_def -lmkl_vml_mc -lmkl_vml_mc3 -lintlc -Wl,--end-group -liomp5 -lpthread

However, the python extension is now failing on specific processor architecture( e.g. Xeon) with a bus error. It does not fail on, say, a Core 2 Duo. The GDB stack is as follows.

#0 0x00002aaaaf25e49f in mkl_blas_ztrsm_lln () from /home/sz9lzw-e/vaone2010/libmkl_mc.so
#1 0x00002aaab1ba2993 in mkl_blas_ztrsm_left_ker () from /home/sz9lzw-e/vaone2010/libmkl_mc3.so
#2 0x00002aaab1ba0411 in mkl_blas_ztrsm_left () from /home/sz9lzw-e/vaone2010/libmkl_mc3.so
#3 0x00002aaab1b98b4b in mkl_blas_xztrsm () from /home/sz9lzw-e/vaone2010/libmkl_mc3.so
#4 0x00002aaaada0bba9 in mkl_blas_xztrsm () from /home/sz9lzw-e/vaone2010/libmkl_intel_thread.so
#5 0x00002aaaadb3ec41 in mkl_blas_ztrsm () from /home/sz9lzw-e/vaone2010/libmkl_intel_thread.so
#6 0x00002aaab49693d3 in __kmp_invoke_microtask () from /home/sz9lzw-e/vaone2010/libiomp5.so
#7 0x00002aaab4946796 in __kmpc_invoke_task_func () from /home/sz9lzw-e/vaone2010/libiomp5.so
#8 0x00002aaab49478e3 in __kmp_launch_thread () from /home/sz9lzw-e/vaone2010/libiomp5.so
#9 0x00002aaab496f347 in ?? () from /home/sz9lzw-e/vaone2010/libiomp5.so
#10 0x00002aaaaae96193 in start_thread () from /lib64/libpthread.so.0
#11 0x00002aaaab3c7dfd in clone () from /lib64/libc.so.6

If I write a "C" version of the calling python, and link to MKL to create a standalone exectutable, that runs fine on all machine architectures (i.e. it is not a known bug in "my" code).

My guess, is that somehow the wrong processor core functions are getting loaded? Anyone have ideas?

GDB "info shared" shows

0x00002aaaad5afff0 0x00002aaaad6f98cc Yes /home/sz9lzw-e/vaone2010/libmkl_intel_lp64.so
0x00002aaaada08690 0x00002aaaae3bdce8 Yes /home/sz9lzw-e/vaone2010/libmkl_intel_thread.so
0x00002aaaaeb86a50 0x00002aaaaec77b10 Yes /home/sz9lzw-e/vaone2010/libmkl_core.so
0x00002aaaaef7c7a0 0x00002aaab0168040 Yes /home/sz9lzw-e/vaone2010/libmkl_mc.so
0x00002aaab0486cf0 0x00002aaab1502bf0 Yes /home/sz9lzw-e/vaone2010/libmkl_p4n.so
0x00002aaab18141c0 0x00002aaab29c7eb0 Yes /home/sz9lzw-e/vaone2010/libmkl_mc3.so
0x00002aaab2cd3e30 0x00002aaab3c47440 Yes /home/sz9lzw-e/vaone2010/libmkl_def.so
0x00002aaab3eb6e10 0x00002aaab40840d0 Yes /home/sz9lzw-e/vaone2010/libmkl_vml_mc.so
0x00002aaab433b9f0 0x00002aaab44fb964 Yes /home/sz9lzw-e/vaone2010/libmkl_vml_mc3.so
0x00002aaab47b0d90 0x00002aaab47d8b60 Yes /home/sz9lzw-e/vaone2010/libintlc.so.5
0x00002aaab4907b80 0x00002aaab4972174 Yes /home/sz9lzw-e/vaone2010/libiomp5.so
0x00002aaab4aac2e0 0x00002aaab4bf4ea0 Yes /home/sz9lzw-e/vaone2010/libimf.so
0x00002aaab4e35b20 0x00002aaab4e8d940 Yes /home/sz9lzw-e/vaone2010/libsvml.so

Linux ...calling MKL 10.2 from Python

Dmitry_B_Intel — Thu, 21 Oct 2010 03:45:09 GMT

Hi,

Is this recent thread releavant to your issue?

Thanks
Dima

Linux ...calling MKL 10.2 from Python

AndrewC — Thu, 21 Oct 2010 15:32:38 GMT

Yes, highly likely.

The original problem was that when the python .so was loaded there was an error

python: symbol lookup error: /home/sz9lzw-e/vaone2010/libmkl_vml_mc3.so: undefined symbol: mkl_serv_mkl_malloc

So I added all the CPU specific libraries to the link line - as the thread you reference indicated this creates more problems.

So I see I either have to create a custom MKL .so or cause the functions to be preloaded.

Linux ...calling MKL 10.2 from Python

AndrewC — Thu, 21 Oct 2010 17:42:08 GMT

Never mind, found it.

Linux ...calling MKL 10.2 from Python

AndrewC — Thu, 21 Oct 2010 20:04:36 GMT

Hi Dima,
I managed to get this problem resiloved using the following technique - as suggested in the other thread

Create a .c file with the following

[bash]#include 
#include 
int start_fn2 () __attribute__ ((constructor));
int start_fn2()
{
        int flags = RTLD_GLOBAL|RTLD_LAZY;
        void *dlh2 = dlopen("libmkl_core.so",flags);
       void *dlh3 = dlopen("libmkl_intel_lp64.so",flags);
      void *dlh4 = dlopen("libmkl_intel_thread.so",flags);
	    printf( "Loaded MKL DLLS  %p %p %pn",dlh2,dlh3,dlh4);fflush(stdout);
			return 0;	
}[/bash]

Compile that file and link it into my python module .so
Add "-z initfirst" to the link line for my python module .so. This was important as some of the other dependent DLL's linked against MKL.
I did need to dlopen all three DLL's, just libmkl_core.so was not enough.

Andrew

Linux ...calling MKL 10.2 from Python

VipinKumar_E_Intel — Tue, 14 Dec 2010 06:49:40 GMT

There is also an article published in the MKL KB with some samples on using MKL in Python applications.

http://software.intel.com/en-us/articles/using-intel-mkl-in-your-python-programs/

--Vipin