topic Linking MKL with JNI. in Intel® oneAPI Math Kernel Library
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186092#M29600
I am using JNI to run a torch function which seems to be using MKL. Note, that I am not using MKL myself but torch does. However, I seem to have trouble with the linking of MKL so I figured someone in here might be able to help me out.
I have created a minimal working example and minimal non working example which can be found here: <A href="https://github.com/jparsert/MinExample" target="_blank">https://github.com/jparsert/MinExample</A>
The following error message is what I get:
INTEL MKL ERROR: /opt/intel/mkl/lib/intel64/libmkl_vml_avx2.so: undefined symbol: mkl_lapack_dspevd.
Intel MKL FATAL ERROR: cannot load libmkl_vml_avx2.so or libmkl_vml_def.so.
The Readme goes into a bit of detail and shows how to reproduce the problem. Anyways the core points being:
1. Everything seems to work fine (compilation and runnning) as long as I stay completely in C++ this is the workingExampleWithoutJNI in the above link.
2. When I invoke a native function from java that does simple Torch things that do not require mkl everything also works.
3. When I invoke a native function from java that uses a torch function that depends on an MKL function (so it seems) I get an INTEL MKL ERROR … undefined symbol: mkl_lapack_dspevd
So basically, when running a native function from JNI the mkl/lapack library seems to not be loaded properly, while when running it with CPP it seems to work.
I see how this may be more of a JNI problem than a MKL problem but maybe someone has a guess where I may be making a mistake.Wed, 10 Jun 2020 12:59:24 GMTParsert__Julian2020-06-10T12:59:24ZLinking MKL with JNI.
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186092#M29600
I am using JNI to run a torch function which seems to be using MKL. Note, that I am not using MKL myself but torch does. However, I seem to have trouble with the linking of MKL so I figured someone in here might be able to help me out.
I have created a minimal working example and minimal non working example which can be found here: <A href="https://github.com/jparsert/MinExample" target="_blank">https://github.com/jparsert/MinExample</A>
The following error message is what I get:
INTEL MKL ERROR: /opt/intel/mkl/lib/intel64/libmkl_vml_avx2.so: undefined symbol: mkl_lapack_dspevd.
Intel MKL FATAL ERROR: cannot load libmkl_vml_avx2.so or libmkl_vml_def.so.
The Readme goes into a bit of detail and shows how to reproduce the problem. Anyways the core points being:
1. Everything seems to work fine (compilation and runnning) as long as I stay completely in C++ this is the workingExampleWithoutJNI in the above link.
2. When I invoke a native function from java that does simple Torch things that do not require mkl everything also works.
3. When I invoke a native function from java that uses a torch function that depends on an MKL function (so it seems) I get an INTEL MKL ERROR … undefined symbol: mkl_lapack_dspevd
So basically, when running a native function from JNI the mkl/lapack library seems to not be loaded properly, while when running it with CPP it seems to work.
I see how this may be more of a JNI problem than a MKL problem but maybe someone has a guess where I may be making a mistake.Wed, 10 Jun 2020 12:59:24 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186092#M29600Parsert__Julian2020-06-10T12:59:24Zhow did you build the case?
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186093#M29601
<P>how did you build the case? did you check the mkl linker adviser first?</P>Thu, 11 Jun 2020 07:28:45 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186093#M29601Gennady_F_Intel2020-06-11T07:28:45ZWhat is the mkl linker
https://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186094#M29602
What is the mkl linker adviser?
I actually "fixed" the problem. And by fixed I mean I found a horrible workaround which I should not have to be doing but it works. I used the ldd tool to see which MKL shared libraries the executable requires and then I added these lines to my code:
void *handle = dlopen("libmkl_core.so", RTLD_LAZY | RTLD_GLOBAL);
void *handle1 = dlopen("libmkl_gnu_thread.so", RTLD_LAZY | RTLD_GLOBAL);
void *handle2 = dlopen("libmkl_intel_lp64.so", RTLD_LAZY | RTLD_GLOBAL);
I would think that this is a JNI loading problem but the ldd tool revealed that my library loads 50 other shared libraries and JNI does not have any trouble loading these. The problem seems to occur only with mkl.Thu, 11 Jun 2020 08:53:33 GMThttps://community.intel.com/t5/Intel-oneAPI-Math-Kernel-Library/Linking-MKL-with-JNI/m-p/1186094#M29602Parsert__Julian2020-06-11T08:53:33Z