Compiling numpy & scipy with Intel MKL in windows 10

[murali_524]

We are facing some issues while following the instruction in following page.

   Numpy/Scipy with Intel® MKL and Intel® Compilers

 We have installed the intel C and fortran compilers and not the full package. [ OneAPI]

Additionally we are consuming the MKL dependencies which are already part of our product.

We facing some error when trying to compile numpy from source.

 Can you point to us where we went wrong.

 Also can you point to already compiled official numpy and scipy packages for windows.

 Terminal log:

>python setup.py config --compiler=intelem build_clib --compiler=intelem build_ext --compiler=intelem install
Running from numpy source directory.
Cythonizing sources
numpy/random\_bounded_integers.pxd.in has not changed
numpy/random\bit_generator.pyx has not changed
numpy/random\mtrand.pyx has not changed
numpy/random\_bounded_integers.pyx has not changed
numpy/random\_bounded_integers.pyx.in has not changed
numpy/random\_common.pyx has not changed
numpy/random\_generator.pyx has not changed
numpy/random\_mt19937.pyx has not changed
numpy/random\_pcg64.pyx has not changed
numpy/random\_philox.pyx has not changed
numpy/random\_sfc64.pyx has not changed
blas_opt_info:
blas_mkl_info:
No module named 'numpy.distutils._msvccompiler' in numpy.distutils; trying from distutils
customize MSVCCompiler
FOUND:
libraries = ['mkl_rt', 'mkl_rt']
library_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\lib\\x64\\Release']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\include']

FOUND:
libraries = ['mkl_rt', 'mkl_rt']
library_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\lib\\x64\\Release']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\include']

lapack_opt_info:
lapack_mkl_info:
FOUND:
libraries = ['mkl_rt', 'mkl_rt']
library_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\lib\\x64\\Release']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\include']

FOUND:
libraries = ['mkl_rt', 'mkl_rt']
library_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\lib\\x64\\Release']
define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
include_dirs = ['C:\\Workspace\\Numpy+MKL\\MKL\\2018.1.156.0\\include']

C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running config
running build_clib
running build_src
build_src
building py_modules sources
building library "npymath" sources
Could not locate executable icc
Could not locate executable ecc
Could not locate executable g77
Could not locate executable f77
Could not locate executable C:\Program
Intel(R) C++ Intel(R) 64 Compiler Classic for applications running on IA-32, Version 2021.2.0 Build 20210228_000000
Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

icl: command line warning #10006: ignoring unknown option '/g'
icl: command line warning #10006: ignoring unknown option '/fPIC'
icl: command line warning #10159: invalid argument for option '/fp'
icl: command line warning #10161: unrecognized source type 'strict'; object file assumed
icl: command line warning #10006: ignoring unknown option '/fomit-frame-pointer'
icl: command line warning #10006: ignoring unknown option '/xhost'
icl: command line warning #10006: ignoring unknown option '/qopenmp'
icl: error #10114: Microsoft Visual C++ not found in path
Traceback (most recent call last):
File "setup.py", line 499, in <module>
setup_package()
File "setup.py", line 491, in setup_package
setup(**metadata)
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\core.py", line 169, in setup
return old_setup(**new_attr)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\site-packages\setuptools\__init__.py", line 153, in setup
return distutils.core.setup(**attrs)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\core.py", line 148, in setup
dist.run_commands()
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\dist.py", line 966, in run_commands
self.run_command(cmd)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\command\build_clib.py", line 80, in run
self.run_command('build_src')
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python37_64\lib\distutils\dist.py", line 985, in run_command
cmd_obj.run()
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\command\build_src.py", line 144, in run
self.build_sources()
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\command\build_src.py", line 155, in build_sources
self.build_library_sources(*libname_info)
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\command\build_src.py", line 288, in build_library_sources
sources = self.generate_sources(sources, (lib_name, build_info))
File "D:\DS\Algo\numpy\new\numpy-1.19.2\numpy\distutils\command\build_src.py", line 378, in generate_sources
source = func(extension, build_dir)
File "numpy\core\setup.py", line 655, in get_mathlib_info
raise RuntimeError("Broken toolchain: cannot link a simple C program")
RuntimeError: Broken toolchain: cannot link a simple C program

[mecej4]

The Intel link that you listed says, as of 2017, that the instructions for building NumPY are outdated, and that prebuilt packages are now available.

The error messages suggest that Linux build commands are being used without modification (or insufficient modification) in Windows. 

[murali_524]

Can you just point us where to get prebuilt packages of numpy and scipy. Can we get some official packages free to use.

[MattHouston]

Perhaps could these outdated instructions be updated ?

[ArunJ_Intel]

Hi murali_524,

You can install get prebuilt packages of numpy and scipy free to use through pip and conda channels.

You could install latest versions of these packages with pip/conda package manager if you have either pip or anaconda installed. Please find the multiple options available, with installation commands in the below links

https://anaconda.org/intel/scipy

https://anaconda.org/intel/numpy

Alternatively you could also get these packages along with other intel optimized python packages by downloading the intel distribution of python from the below link.

https://software.intel.com/content/www/us/en/develop/articles/oneapi-standalone-components.html#python

All the mentioned options (conda,pip and intel distribution of python) are available for free)

Thanks

Arun

[ArunJ_Intel]

Hi murali_524,

Have you tried out the prebuilt packages, are there any further queries regarding these. If this resolves your issue, make sure to accept this as a solution. This would help others with similar issue. 

Thanks

Arun

[ArunJ_Intel]

Hi murali_524,

As we have not heard back from you for sometime now we assume the issue has been resolved. If you need any additional information, please submit a new question as this thread will no longer be monitored.

Arun