MKL with Spark

[Jacques_J_]

Hello,

I'm trying to use MKL with Spark using netlib-java. I included the folder containing the dlls in the Path variable and specified the following option to the JVM : -Dcom.github.fommil.netlib.BLAS=mkl_rt.dll

However, it doesn't work and I still get the following warnings :

17/08/10 14:22:09 WARN BLAS: Failed to load implementation from: mkl_rt.dll
17/08/10 14:22:09 WARN BLAS: Using the fallback implementation.

Any help would be greatly appreciated

[Ying_H_Intel]

Hi Jacques,

Do you use spark under windows?   mkl_rt.dll is window library, where mkl_rt.so is Linux library.  As it is dynamic library, you may add some environment LD_LIBRARY_PATH  or run script  source /opt/intel/mkl/mklvars.sh intel64  before run your spark application

what kind of application do you like to run based on MKL?  I may recommend one of data analysis accelerate library DAAL , which is based on MKL, have java /c/python interface,  and also work on Spark and MLlib.

get free DAAL from  https://software.intel.com/en-us/intel-daal

DAAL training webpage:

https://software.intel.com/en-us/intel-daal-support/training

DAAL Scala Spark sample  download from

https://software.intel.com/en-us/product-code-samples

Best Regards,

Ying