Error building NumPy with MKL on OSX with Python 3.3

[MattDMo]

I just asked this question on StackOverflow, so feel free to answer either place.

Background:

Using a 2012 iMac with a 2nd-gen Core i7 processor, I'm trying to build NumPy 1.7.0 (and eventually SciPy) on OSX 10.8.3 linked to the MKL included in the evaluation versions of C++ Composer XE 2013 and Fortran Composer XE 2013 for OSX. I am following this article on Intel's site (loosely adapted, as it's not very specific in some cases) for settings, compiler flags, etc.

Problem

Using either:

[bash]$ python3 setup.py config build --compiler=intelem --fcompiler=intelem[/bash]

or:

[bash]$ python3 setup.py config --compiler=intelem --fcompiler=intelem build_clib --compiler=intelem --fcompiler=intelem build_ext --compiler=intelem --fcompiler=intelem[/bash]

I get the same error:

[bash]

running build_clib customize IntelEM64TCCompiler customize IntelEM64TCCompiler using build_clib running build_ext customize IntelEM64TCCompiler customize IntelEM64TCCompiler using build_ext building 'numpy.core._dummy' extension compiling C sources C compiler: icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include compile options: '-Inumpy/core/include -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/include/numpy -Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/src/multiarray -Ibuild/src.macosx-10.8-x86_64-3.3/numpy/core/src/umath -c' icc: numpy/core/src/dummymodule.c icc: command line warning #10121: overriding '-marchcorei7' with '-xhost' icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include -shared build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so icc: command line warning #10121: overriding '-marchcorei7' with '-xhost' icc: command line warning #10006: ignoring unknown option '-shared' Undefined symbols for architecture x86_64: "_PyModule_Create2", referenced from: _PyInit__dummy in dummymodule.o "_main", referenced from: start in crt1.10.6.o ld: symbol(s) not found for architecture x86_64 icc: command line warning #10121: overriding '-marchcorei7' with '-xhost' icc: command line warning #10006: ignoring unknown option '-shared' Undefined symbols for architecture x86_64: "_PyModule_Create2", referenced from: _PyInit__dummy in dummymodule.o "_main", referenced from: start in crt1.10.6.o ld: symbol(s) not found for architecture x86_64 error: Command "icc -m64 -march=corei7 -O3 -g -fPIC -fp-model strict -fomit-frame-pointer -openmp -xhost -DMKL_ILP64 -I/opt/intel/mkl/include -shared build/temp.macosx-10.8-x86_64-3.3/numpy/core/src/dummymodule.o -Lbuild/temp.macosx-10.8-x86_64-3.3 -o build/lib.macosx-10.8-x86_64-3.3/numpy/core/_dummy.so" failed with exit status 1 [/bash][/bash]

My LD_LIBRARY_PATH and DYLD_LIBRARY_PATH are both:

[bash]/opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib[/bash]

my DYLD_FRAMEWORK_PATH is:

[bash]/opt/local/Library/Frameworks:/Library/Frameworks:/System/Library/Frameworks[/bash]

and my site.cfg is:

[bash] [DEFAULT] library_dirs=/opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib include_dirs=/opt/intel/include:/opt/intel/include/intel64:/opt/intel/mkl/include:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m:/opt/local/include:/usr/local/include:/usr/include [mkl] library_dirs=/opt/intel/lib/intel64:/opt/intel/lib:/opt/intel/mkl/lib:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/lib:/opt/local/lib:/usr/local/lib:/usr/lib include_dirs=/opt/intel/include:/opt/intel/include/intel64:/opt/intel/mkl/include:/opt/local/Library/Frameworks/Python.framework/Versions/3.3/include/python3.3m:/opt/local/include:/usr/local/include:/usr/include mkl_libs= mkl_rt lapack_libs=mkl_lapack[/bash]

Symlinks to icc and ifort are in /usr/bin and are functional. The contents of dummymodule.c are available in the numpy repository on GitHub. I'm not a C programmer, so I can't immediately see what's wrong. I've gotten this error using both MacPorts python 3.3.0 (using the above settings) and with the python.org 3.3.0, with settings modified to point to it instead of /opt/local.

Plea

Can anyone help figure this out? Does the PYTHON_PATH need to be set, and if so to what? I'm not sure where "_PyModule_Create2" is set, so am I missing a library or include path? I'm really stuck, and I really want to get this to work!

Thank you for your time!

Matt

[Zhang_Z_Intel]

The procedures of building NumPy/SciPy with MKL described in this article are only tested for Python 2.6 and 2.7. I haven't got time to try it for Python 3.x.

The undefined symbol "_PyModule_Create2" surely does not come from MKL. I guess this is a problem of your Python environment settings. I'd suggest you build with Python 2.7. If you insist on using Python 3.3, then please first try to build it without using MKL. If it still doesn't work then you'd better escalate the issue to the NumPy community.

Thanks,

Zhang

[MattDMo]

Hi Zhang,

I know that numpy will build with Python 3.3 (I haven't tested it myself yet, but I will) on OSX, and will build with 3.3 and MKL on Windows, so theoretically it's possible. Thanks for your input!

Matt