Undefined references to mkl objects

[Sresht__Vishnu]

I'm attempting to compile OpenCV with Intel MKL 2017.0 on Ubuntu 16.04.1 with , but I keep running up against the following errors at compile time:

/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_lp64_ccsr0ttluc__svout_seq'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_zlantp'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_sdia1ttluf__mvout_par'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_lp64_ccsr1ntuuf__mvout_par'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_blas_xdgemv'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_ccsr1ntuuf__smout_par'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_sparse_d_csr_ng_n_spmmd_ker_i4'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_ccoo0nhluc__mvout_par'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_sequential.so: undefined reference to `mkl_spblas_lp64_dcsr1ttluf__svout_seq'
/opt/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64_lin/libmkl_intel_lp64.so: undefined reference to `mkl_lapack_dgelsd'

I have installed Intel MKL and I have the following in my zsh startup script:

source /opt/intel/mkl/bin/mklvars.sh intel64

I also have the following in /etc/ld.so.conf.d/intel_mkl.conf :

/opt/intel/lib/intel64
/opt/intel/mkl/lib/intel64

CMake manages to find MKL and reports:

-- Found MKL 2017.0.1 at: /opt/intel/mkl
-- LAPACK(MKL): LAPACK_LIBRARIES: /opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;-lpthread;-lm;-ldl
-- LAPACK(MKL): Support is enabled.

CMake also lists:

--     Use Intel VA-API/OpenCL:     NO
--     Use Lapack:                  YES (/opt/intel/mkl/lib/intel64/libmkl_core.so /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so /opt/intel/mkl/lib/intel64/libmkl_sequential.so /opt/intel/mkl/lib/intel64/libmkl_core.so /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so /opt/intel/mkl/lib/intel64/libmkl_sequential.so /opt/intel/mkl/lib/intel64/libmkl_core.so /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so /opt/intel/mkl/lib/intel64/libmkl_sequential.so -lpthread -lm -ldl)
--     Use Eigen:                   NO
--     Use Cuda:                    YES (ver 8.0)
--     Use OpenCL:                  YES

What am I missing? Are there any additional compile flags I should be specifying?

[Jing_Xu]

Hi,

Could you store the following 3 when compiling OpenCV against MKL, and then share with us?

1. history of commands, step-by-step.

2. environment variables (before execution of cmake). They could be exported to file with the following command on Linux: env > filename.

3. cmake history. They could be redirected to file with the following command on Linux: cmake ......... 2>&1 | tee filename.

[kobe_y_]

Hi

I have the same problem, I just workaround by add flag "-D WITH_LAPACK=OFF".

[Sresht__Vishnu]

Jing X. (Intel) wrote:

Hi,

Could you store the following 3 when compiling OpenCV against MKL, and then share with us?

1. history of commands, step-by-step.

2. environment variables (before execution of cmake). They could be exported to file with the following command on Linux: env > filename.

3. cmake history. They could be redirected to file with the following command on Linux: cmake ......... 2>&1 | tee filename.

Hi Jing,

Here's what I did:

1. Installed MKL using the instructions here: https://software.intel.com/en-us/articles/intel-mkl-113-install-guide
2. Added the following to (a new) file /etc/ld.so.conf.d/intel_mkl.conf: 

   /opt/intel/lib/intel64
   /opt/intel/mkl/lib/intel64

3. Added the following to my .zshrc file:
   source /opt/intel/mkl/bin/mklvars.sh intel64
4. Clone the OpenCL repository at https://github.com/opencv/opencv.git
5. Clone the OpenCL extras repository at https://github.com/opencv/opencv_contrib.git

6. Run CMAKE from within a freshly created 'release' directory within the cloned opencv repository:

   cd opencv 

   mkdir release  

   cd release

   cmake -D CMAKE_BUILD_TYPE=RELEASE \ 

         -D CMAKE_INSTALL_PREFIX=/opt/opencv \ 

         -D WITH_CUDA=ON \ 

         -D ENABLE_FAST_MATH=1 \ 

         -D CUDA_FAST_MATH=1 \ 

         -D WITH_CUBLAS=1 ..  \ 

         -D INSTALL_PYTHON_EXAMPLES=ON \ 

         -D WITH_OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules \ 

         -D WITH_TBB=ON \ 

         -D BUILD_NEW_PYTHON_SUPPORT=ON \ 

         -D WITH_V4L=ON \ 

         -D INSTALL_C_EXAMPLES=ON \ 

         -D BUILD_EXAMPLES=ON \ 

         -D WITH_QT=ON \ 

         -D WITH_OPENGL=ON \ 

         .. 2>&1 | tee build_output.log
7. make -j8 > >(tee make_output.log) 2> >(tee make_errors.log >&2)
8. ERROR

I have attached files listing my environment variables, cmake output, build output and build errors to this post. I hope they help you help me figure out what's going on!

[Sresht__Vishnu]

kobe yu wrote:

Hi

I have the same problem, I just workaround by add flag "-D WITH_LAPACK=OFF".

Hi Kobe,

That's an excellent idea and it totally works! However, I would still like to know why I'm unable to link Intel's lapack to OpenCV and I'm hoping you, or someone else, can help me figure this out.

[Sresht__Vishnu]

I found a fix that allows you to compile OpenCV using MKL.

All of those undefined references were references to objects in libmkl_core.so. Even though this library is specified in the link line, gcc on Ubuntu 16.04 doesn't look inside it when searching for these items.

The fix was to specify -Wl,--no-as-needed in the link line before the Intel MKL libraries. This can be accomplished in a CMAKE-based project by setting the variable CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=-Wl,--no-as-needed in CMakeCache.txt

[Jing_Xu]

Great to hear that.

Thank you for sharing the solution.

[Ying_H_Intel]

Hi Vishnu,

thanks much for the sharing.  We can't reproduce the problem under Linux.  But the issue do related to the system and mkl library.

all unreferenced symbols like mkl_lapack_zlantp  are defined in libmkl_core.so

[yhu5_new@hsw-ep01 cmake]$ nm  /opt/intel/mkl/lib/intel64/libmkl_core.so |grep mkl_lapack_zlantp
0000000000a14d20 T mkl_lapack_zlantp

"libmkl_intel_lp64 libmkl_sequential libmkl_core " are supposed to be used together and here in the link line, the combination were repeated several times, but miss mkl_core at last one.

opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;/opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_sequential.so;-lpthread;-lm;-ldl

So It seems there is some problem in FINDLAPACK.cmake processing( OpenCVFindMKL.cmake).  the fix is  CMAKE_SHARED_LINKER_FLAGS_RELEASE:STRING=-Wl,--no-as-needed

or like the article https://software.intel.com/en-us/articles/symbol-lookup-error-when-linking-intel-mkl-with-gcc-on-ubuntu

If possible, could you please try Set environment variable >export  LD_PRELOAD=/path/to/libmkl_core.so and see if it can work around?

Thanks

Ying

[Sresht__Vishnu]

Hi Ying,

Thank you for that alternate solution. Yes, setting LD_PRELOAD=/path/to/libmkl_core.so also works. So apparently, there is some issue when it comes to the combination of gcc, mkl, and ubuntu. Fortunately the two workarounds above seem to work.