Problem with numpy and MKL (cannot find __kmpc_reduce_nowait)

[winterforest]

I apologize for another numpy question.
I read many articles but I could not find the solution for my problem.

I am trying to install numpy with MKL and install went well.
However, whenever I try to import numpy, a following error message shows:

File "", line 1, in
File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line 137, in
import add_newdocs
File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 9, in
from numpy.lib import add_newdoc
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 13, in
from polynomial import *
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py", line 17, in
from numpy.linalg import eigvals, lstsq
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py", line 48, in
from linalg import *
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py", line 23, in
from numpy.linalg import lapack_lite
ImportError: /opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64/libmkl_intel_thread.so: undefined symbol: __kmpc_reduce_nowait

I think this is related with openmp, but I cannot find a solution.

Here is my environments:

* Ubuntu 11.10
* Numpy1.6.1
* python 2.7.2
* libgomp.so.1.0.0
* core i7 (quad)
* Intel Composer1.7.256

* LD_LIBRARY_PATH = /usr/lib:/opt/intel/composer_xe_2011_sp1.7.256/compiler/include:
/opt/intel/composer_xe_2011_sp1.7.256/compiler/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/ipp/../compiler/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/ipp/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/compiler/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/tbb/lib/intel64//cc4.1.0_libc2.4_kernel2.6.16.21:
/opt/intel/composer_xe_2011_sp1.7.256/debugger/lib/intel64:
/opt/intel/composer_xe_2011_sp1.7.256/mpirt/lib/intel64
(obtained by source /opt/intel/composer_xe_2011_sp1.7.256/bin/compilervars.sh intel64 and . /opt/intel/composer_xe_2011_sp1.7.256/mkl/bin/intel64/mklvars_intel64.sh)

*site.cfg
[mkl]
library_dirs = /opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64
include_dirs = /opt/intel/composer_xe_2011_sp1.7.256/mkl/include
mkl_libs = mkl_intel_lp64,mkl_intel_thread,mkl_core,mkl_mc
(I tried mkl_libs = mkl_rt as well)

complie commad
icc -openmp -lpthread -O2 -g -xSSE4.2 -DMKL_ILP64 -fPIC

Any help would be very appreciate.

sincerely.

[Gennady_F_Intel]

two notes regarding this issue:

- please try to explicitly addlibiomp5 to your cfg files

- compiler option -DMKL_ILP64 is not compatible with LP64 libraries you are using.Linking of an application compiled with the -DMKL_ILP64 option to the LP64 librariesmay result in unpredictable consequences and erroneous output.

please remove this option and check how it will works.

--Gennady

[VipinKumar_E_Intel]

Can you try setting LD_RUN_PATH pointing to /opt/intel/composer_xe_2011_sp1.7.256/compiler/lib/intel64 ?

--Vipin

[winterforest]

Thanks for quick replies!

I tried all of them, but it still does not work.

1) settinh ld_run_path did not work..
2) removing compile option did not work (icc -openmp -lpthread -O2 -g -xSSE4.2 -fPIC)
3) adding iomp5 did not work. As for libiomp5.so and libiomp5.a, I had to copy it from compiler/lib/intel64/ to mkl/lib/intel64/, since it was not at mkl lib.
lapack library seems going well.


lapack_mkl_info:
FOUND:
libraries = ['mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
library_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/include']

FOUND:
libraries = ['mkl_lapack', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
library_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/include']

FOUND:
libraries = ['mkl_lapack', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'iomp5', 'pthread']
library_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64']
define_macros = [('SCIPY_MKL_H', None)]
include_dirs = ['/opt/intel/composer_xe_2011_sp1.7.256/mkl/include']

still I have a message
from numpy.linalg import lapack_lite
ImportError: /opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64/libmkl_intel_thread.so: undefined symbol: __kmpc_reduce_nowait

I wonder why kmpc_reduce_nowait is not defined in libmkl_intel_thread.so.
Also how I can change import libiomp5.so instead of libmkl_intel_thread.

I apologize for bothering you and appreciate your help!

[TimP]

In case it wasn't clear from the previous replies, libmkl_intel_thread makes the OpenMP implementation library calls for MKL, but those functions are defined in libiomp5. You must link against that library, and (if linking the dynamic version) have it available at run time. libiomp5 also supports compiler OpenMP and auto-parallel, and you would not want multiple implementations of the functions in case OpenMP is in use both from within MKL and by compiled code. It's not as if MKL was designed only for your numpy usage.

[winterforest]

Thank you TimP. I slightly understood the structure of MKL and OpenMP.

After configuring LD_RUN_PATH, python path, and install configuration, still I get the same error.
As a light linux-user, I wish everything would work by following an instruction and user guide..
(I referred here: http://software.intel.com/en-us/articles/numpy-user-note/ and was helped a lot by install_var*)
Perhaps your MKL is not for me;;

Anyway, thank all of you for a help!

sincerely.

[TimP]

In order for your application to find the MKL .so files at run time, the usual mechanism is to set LD_LIBRARY_PATH, as would be done by sourceing the mklvars script, or the compilervars script if your MKL was provided with an Intel compiler.

[winterforest]

Thank you! Timp. I finally succeeded to install it.

I edited site.cfg like this:

[mkl]
library_dirs = /opt/intel/composer_xe_2011_sp1.7.256/mkl/lib/intel64
include_dirs = /opt/intel/composer_xe_2011_sp1.7.256/mkl/include
mkl_libs = mkl_def, mkl_intel_lp64,mkl_intel_thread,mkl_core
lapack_libs =

mkl_def was the key.
I am looking forward to using it.

yours sincerely