NumPy ImportError: libmkl_avx.so: undefined symbol: ownLastTriangle_64fc

[MattDMo]

I'm running Ubuntu 13.04, and have the latest versions of the MKL, icc, and ifort. I'm trying to build NumPy 1.8.0beta2 and have been running into an unusual error. The package builds and installs just fine, but when I run "import numpy as np" in the Python 3.3.2 (gcc-built) interpreter, I get the following traceback:

[python]

>>> import numpy as np
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.3/site-packages/numpy/__init__.py", line 153, in <module>
    from . import add_newdocs
  File "/usr/local/lib/python3.3/site-packages/numpy/add_newdocs.py", line 13, in <module>
    from numpy.lib import add_newdoc
  File "/usr/local/lib/python3.3/site-packages/numpy/lib/__init__.py", line 18, in <module>
    from .polynomial import *
  File "/usr/local/lib/python3.3/site-packages/numpy/lib/polynomial.py", line 19, in <module>
    from numpy.linalg import eigvals, lstsq, inv
  File "/usr/local/lib/python3.3/site-packages/numpy/linalg/__init__.py", line 50, in <module>
    from .linalg import *
  File "/usr/local/lib/python3.3/site-packages/numpy/linalg/linalg.py", line 29, in <module>
    from numpy.linalg import lapack_lite, _umath_linalg
ImportError: /opt/intel/mkl/lib/intel64/libmkl_avx.so: undefined symbol: ownLastTriangle_64fc
>>>

[/python]

My site.cfg is as follows:

[plain]

[DEFAULT]
library_dirs = /opt/intel/lib:/opt/intel/lib/intel64:/opt/intel/mkl/lib:/opt/intel/mkl/lib/intel64:/usr/local/lib/python3.3
include_dirs = /opt/intel/include/intel64:/opt/intel/include:/opt/intel/mkl/include:/opt/intel/mkl/include/intel64/lp64:/usr/local/include/python3.3m

[mkl]
library_dirs = /opt/intel/lib:/opt/intel/lib/intel64:/opt/intel/mkl/lib:/opt/intel/mkl/lib/intel64
include_dirs = /opt/intel/include/intel64:/opt/intel/include:/opt/intel/mkl/include:/opt/intel/mkl/include/intel64/lp64
mkl_libs = mkl_rt,mkl_intel_lp64,mkl_intel_thread,mkl_core,mkl_def,mkl_avx,mkl_vml_def,mkl_lapack95_lp64,mkl_blas95_lp64
lapack_libs = mkl_lapack95_lp64
blas_libs = mkl_blas95_lp64

[blas]
library_dirs = /opt/intel/lib:/opt/intel/lib/intel64:/opt/intel/mkl/lib:/opt/intel/mkl/lib/intel64
include_dirs = /opt/intel/include/intel64:/opt/intel/include:/opt/intel/mkl/include:/opt/intel/mkl/include/intel64/lp64
blas_libs = mkl_blas95_lp64

[lapack]
library_dirs = /opt/intel/lib:/opt/intel/lib/intel64:/opt/intel/mkl/lib:/opt/intel/mkl/lib/intel64
include_dirs = /opt/intel/include/intel64:/opt/intel/include:/opt/intel/mkl/include:/opt/intel/mkl/include/intel64/lp64
lapack_libs = mkl_lapack95_lp64
[/plain]

If I just use mkl_libs = mkl_rt I get an error complaining that libmkl_avx and libmkl_def can't be found, and if I just have those three I get the same error as I'm getting now.

I changed numpy/distutils/intelccompiler.py to have the following:

[python]

self.cc_exe = 'icc -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -mkl -m64 -DMKL_LP64 -L/opt/intel/lib -L/opt/intel/lib/intel64 -L/opt/intel/mkl/lib -L/opt/intel/mkl/lib/intel64 -L/usr/local/lib/python3.3 -I/opt/intel/include/intel64 -I/opt/intel/include -I/opt/intel/mkl/include -I/opt/intel/mkl/include/intel64/lp64 -I/usr/local/include/python3.3m'

[/python]

and I changed numpy/distutils/fcompiler/intel.py in similar fashion:

[python]

def get_flags_opt(self):
    return [' -O3 -mkl -xhost -openmp -fp-model strict -fPIC -m64 -DMKL_LP64 -L/opt/intel/lib -L/opt/intel/lib/intel64 -L/opt/intel/mkl/lib -L/opt/intel/mkl/lib/intel64 -L/usr/local/lib/python3.3 -I/opt/intel/include/intel64 -I/opt/intel/include -I/opt/intel/mkl/include -I/opt/intel/mkl/include/intel64/lp64 -I/usr/local/include/python3.3m']

[/python]

Finally, I commented out lines 953-956 in numpy/distutils/system_info.py where the pthread library is automatically added, as I want to use Intel's threading library included in the MKL.

I googled the error I'm getting, and there were literally zero hits. I'm not sure where this symbol is defined, so I don't know how to link to the required library. Can anyone offer any assistance?

Thanks,

Matt

[Noah_C_Intel]

Hi, have you taken a look at this article? http://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl

This article walks through everything and will probably anticipate future issues as well. Another good link to bookmark is the link line advisor which is always a good tool for general build and run: http://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

Take a look at the NumPy article, go through all those steps and let me know how it shakes out. From there we will have a baseline to rule out many different issues.

[Zhang_Z_Intel]

Jared,

The article you referenced has recently been updated for the latest NumPy and MKL releases. But it only covers the situation of using Intel tool chain (Intel compiler and Intel OpenMP threading). You are using GNU compilers and GNU OpenMP. So we need some more investigation.

Please post your 'site.cfg', as well as your compiler configuration files (the relevant ones found under the distutil directory of the NumPy source).

Thanks.

[Jared_B_]

Noah,

That article is quite a bit dated for Intel MKL and even NumPy. I just ran into the same issue above where Intel MKL is having undefined references to the ownLastTriagle_64fc. Confirming against the shared library:

[bash]

$ nm libmkl_avx.so | grep ownLastTriangle_64fc

U ownLastTriangle_64fc

[/bash]

Although I am using the GNU compilers (gcc,g++,gfortran) for this particular build of NumPy. The line to compile essentially looks as:

[bash]gcc -m64 -mavx -march=native -mtune=native -fopenmp -I${MKLROOT}/include ...[/bash]

The link like would look something like the following:

[bash]gcc -L${MKLROOT}/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lmkl_avx -lmkl_def -ldl -lpthread -lm -Wl,-rpath,${MKLROOT}/lib/intel64 ...[/bash]

The previous link line which I recieved the error earlier was like:

[bash] gcc -L${MKLROOT}/lib/intel -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -ldl -lpthread -lm -Wl,-rpath,${MKLROOT}/lib/intel64 ...[\bash]

works during the import, but then actually starting a test with NumPy, MKL issues a fatal error:

Python 2.7.6 (default, Mar 6 2014, 09:13:15)

>>>import numpy 
>>> numpy.test() 
Running unit tests for numpy NumPy version 1.8.0 NumPy is installed in /apps/python/gnu/2.7.6/lib/python2.7/site-packages/numpy Python version 2.7.6 (default, Mar 6 2014, 09:13:15) [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] nose version 1.3.0 ................. 
Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

Which is why I lead to the method of adding libmkl_avx.so and libmkl_def.so to the link line and then get the undefined reference. I didn't have this problem when I was using MKL 10.3, only when I started moving code to MKL 11 It appears that the string can be located in the files following:

$ grep ownLastTriangle_64fc *

Binary file libmkl_avx2.so matches 
Binary file libmkl_avx.so matches 
Binary file libmkl_core.a matches 
Binary file libmkl_def.so matches 
Binary file libmkl_mc3.so matches 
Binary file libmkl_mc.so matches 
Binary file libmkl_p4n.so matches

Changing the link line to use the single runtime library (libmkl_rt.so) issues the same MKL FATAL ERROR.

Have any ideas where to go from here?

[Jared_B_]

site.cfg:

[DEFAULT]
library_dirs = /lib64:/usr/lib64:/usr/local/lib
include_dirs = /usr/include:/usr/local/include
libraries = dl, pthread, m

[mkl]
library_dirs = /apps/INTEL/2013/mkl/lib/intel64
include_dirs = /apps/INTEL/2013/mkl/include
mkl_libs = mkl_gf_lp64, mkl_gnu_thread, mkl_core
lapack_libs = 

The compiler happens as the following, I don't necessarily modify the files under distutil directory. I just append certain flags using the environment variables here. The compiler files under distutils are unchanged. I prefer this method rather than editing the *.py files.

export CPPFLAGS="-I$MKLROOT/include"
export CFLAGS="-mavx -m64 -fopenmp -march=native -mtune=native -O3 -fPIC -fomit-frame-pointer"
export CXXFLAGS="-mavx -m64 -fopenmp -march=native -mtune=native -O3 -fPIC -fomit-frame-pointer"
export FFLAGS="-mavx -m64 -fopenmp -march=native -mtune=native -O3 -fPIC -fomit-frame-pointer"
export LDFLAGS="-fopenmp -L$MKLROOT/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -ldl -lpthread -lm -Wl,-rpath,$MKLROOT/lib/intel64"

python setup.py config
python setup.py build
python setup.py install

Then when I try to run the numpy test after the build:

Python 2.7.6 (default, Mar  6 2014, 09:13:15) 
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
>>> numpy.test()
Running unit tests for numpy
NumPy version 1.8.0
NumPy is installed in /apps/python/gnu/2.7.6/lib/python2.7/site-packages/numpy
Python version 2.7.6 (default, Mar  6 2014, 09:13:15) [GCC 4.4.6 20120305 (Red Hat 4.4.6-4)]
nose version 1.3.0
.................
Intel MKL FATAL ERROR: Cannot load libmkl_avx.so or libmkl_def.so.

I make sure I sanitize the NumPy build each time a make a change as well.

[Eloi_Gaudry]

Jared, I don't think that :

gcc -L${MKLROOT}/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -lmkl_avx -lmkl_def -ldl -lpthread -lm -Wl,-rpath,${MKLROOT}/lib/intel64 ...

is correct, but this should be ok:

gcc -L${MKLROOT}/lib/intel64 -lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core -ldl -lpthread -lm -Wl,-rpath,${MKLROOT}/lib/intel64 ...

And, as far as I know, the issue at runtime with numpy is still not solved with Mkl-11.1.2.144...

Any progress from Intel side ?

Eloi

[Hoste__Kenneth]

The page https://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl now has a section "Building with GNU Compiler chain" at the bottom that says to only specify mkl_rt for mkl_libs in site.cfg.

I tried this after running into the same problem being reported here, and it resolved the issue for me (using GCC 4.8.3, imkl 11.1.2.144 and numpy 1.8.0).

[Bicici__Ergun]

I encountered the same error after I removed a remaining .so file from a previous installation of numpy (/usr/local/lib/python3.7/site-packages/numpy/.libs/libopenblasp-r0-34a18dc3.3.7.so). Similar to the suggestion at:

https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/733207

I added mkl_def in site.cfg of numpy.