topic Sounds good! I will give a in Intel® oneAPI Math Kernel Library

SciPy built with Intel MKL Update 2 fails test

mvrht__u3425923 — Wed, 04 Apr 2018 14:08:19 GMT

Hi,

SciPy v1.0.0 built with newly released Intel MKL 2018 Update 2 using GNU gcc/gfortran on Ubuntu 16.04 x64 PC fails standard tests. According to test output (attached), as I understand, some modules not included into final shared library despite that building process ended without errors (log also attached). It even do not start testing process detecting that module is absent:

AttributeError: module 'scipy.spatial.qhull' has no attribute 'ConvexHull'

where qhull.py only loads and imports symbols from dedicated shared library. Problem appeared since MKL 2018 update 1 release, SciPy built with MKL 2018 initial release successfully passes tests.

Steps to reproduce:

1. build scipy:

wget https://github.com/scipy/scipy/archive/v1.0.0.tar.gz
tar -xzf v1.0.0.tar.gz
cd scipy-1.0.0
echo "[mkl]" >> site.cfg
echo "library_dirs = /opt/intel/mkl/lib/intel64" >> site.cfg
echo "include_dirs = /opt/intel/mkl/include" >> site.cfg
echo "mkl_libs = mkl_rt" >> site.cfg
echo "lapack_libs = " >> site.cfg
python3 setup.py build
sudo python3 setup.py install

2. Run standard SciPy test:

python3
>> import scipy as sp
>> sp.test(verbose=3)

I am trying to build custom Docker image. As Intel python distribution contains a lot of packages and has completely non-customizable install it is not a suitable solution, it is too heavy (5Gb), and in fact I still need original MKL for C++ code. I am able to reproduces this error on multiple PC.

thanks for the issue. We will

Gennady_F_Intel — Thu, 05 Apr 2018 03:24:39 GMT

thanks for the issue. We will ask Python developers to investigate this case.

Hi Mvrht,

Ying_H_Intel — Sun, 08 Apr 2018 01:56:24 GMT

Hi Mvrht,

Do set KMP_INIT_AT_FORK=FALSE work around the problem?

if it is same reason from
https://software.intel.com/en-us/forums/intel-math-kernel-library/topic/760830, ;

We will fix in latest next release.

Thanks
Ying

Hi, Ying,

mvrht__u3425923 — Sun, 08 Apr 2018 09:52:01 GMT

Hi, Ying,

no, it does not help. I think it is more compilation time problem than runtime, though I am not 100% sure

Rohit_J_Intel — Tue, 10 Apr 2018 19:03:30 GMT

Hello,

Since your primary concern is having Intel optimized Scipy, and not the complete installation of Intel Distribution for Python, functional in a container, I'd suggest the following:

1. Install miniconda in container (https://conda.io/miniconda.html)
2. <miniconda_installation>/bin/conda create -n intel_core_pkgs -c intel intelpython3_core python=3.6 -y
3. source <miniconda_installation>/bin/activate intel_core_pkgs

When you activate the "intel_core_pkgs" conda-environment (Step.3), you should have Scipy and all the required packages installed in that environment. I'd advise you to take this route since we build Numpy and Scipy using Intel compilers and have made various optimizations to the sources prior to building these modules.

Hope this helps!

Thanks,

Rohit

Hello, Rohit,

mvrht__u3425923 — Wed, 11 Apr 2018 14:34:42 GMT

Hello, Rohit,

thank you for advice, but I guess in such case I will have to install separate MKL for C/C++ code?

Rohit_J_Intel — Wed, 11 Apr 2018 15:04:00 GMT

Hi,

You won't have to; since we build Scipy and Numpy while dynamically linking against MKL, we ship MKL runtimes with the distribution. In your conda-environment, you would have all the MKL runtimes at `python3-config --exec-prefix`/lib directory. Also, if you require MKL's header-files, you can get them by running: `conda install -c intel mkl-include`, from your conda-environment. The header-files would be placed in `python3-config --exec-prefix`/include directory.

Thanks,

Rohit

Sounds good! I will give a

mvrht__u3425923 — Wed, 11 Apr 2018 16:21:25 GMT

Sounds good! I will give a try, thank you!

Hi, the error of compiling

Oleksandr_P_Intel — Thu, 12 Apr 2018 12:56:50 GMT

Hi, the error of compiling qhull.pyx is due to use of Cython 0.28, which broke build of SciPy. See https://github.com/scipy/scipy/pull/8581

The fix from the aforementioned pull request has been incorporated in 1.0.1 sources. Please use those, or use Cython 0.27.3 to build scipy.

Hi,

mvrht__u3425923 — Thu, 12 Apr 2018 20:50:50 GMT

Hi,

haven't noticed that SciPy update. Thank you!