Link error when building numpy with MKL on Windows

[Bo1]

Hi,

I'm now building numpy on my Windows 7 following this guide: https://software.intel.com/en-us/articles/building-numpyscipy-with-intel-mkl-and-intel-fortran-on-windows. I also posted this question under the guide, if it's not proper please delete it. 

I have Intel Parallel Studio XE 2017 installed including MKL, icl and ifort. The version of numpy is the latest one because I've downloaded just now. 

When I did the step 4, complied numpy with the indicated line: 

python setup.py config --compiler=intelemw build_clib --compiler=intelemw build_ext --compiler=intelemw install

then an error was raised, the key part is

LINK : fatal error LNK1104: cannot open file 'libiomp5md.lib'

while actually the file libiomp5md.lib did lie in the directory I configured in site.cfg, and it was found during the process as the following log said. So I have no idea where the problem is.

Could anyone give me some hints? Thanks.

Bo

The log is as follows:

Running from numpy source directory.
blas_opt_info:
blas_mkl_info:
  FOUND:
    libraries = ['mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md']
    library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\mkl\\lib\\intel64_win', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\compiler\\lib\\intel64_win']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\parallel_studio_xe_2017.0.036\\compilers_and_libraries_2017\\windows\\mkl\\include']

  FOUND:
    libraries = ['mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md']
    library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\mkl\\lib\\intel64_win', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\compiler\\lib\\intel64_win']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\parallel_studio_xe_2017.0.036\\compilers_and_libraries_2017\\windows\\mkl\\include']

F2PY Version 2
lapack_opt_info:
openblas_lapack_info:
  libraries openblas not found in ['C:\\Python27\\lib', 'C:\\', 'C:\\Python27\\libs']
  NOT AVAILABLE

lapack_mkl_info:
mkl_info:
  FOUND:
    libraries = ['mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md']
    library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\mkl\\lib\\intel64_win', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\compiler\\lib\\intel64_win']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\parallel_studio_xe_2017.0.036\\compilers_and_libraries_2017\\windows\\mkl\\include']

  FOUND:
    libraries = ['mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md', 'mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md']
    library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\mkl\\lib\\intel64_win', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\compiler\\lib\\intel64_win']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\parallel_studio_xe_2017.0.036\\compilers_and_libraries_2017\\windows\\mkl\\include']

  FOUND:
    libraries = ['mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md', 'mkl_lapack95_lp64', 'mkl_blas95_lp64', 'mkl_intel_lp64', 'mkl_intel_thread', 'mkl_core', 'libiomp5md']
    library_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\mkl\\lib\\intel64_win', 'C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2017.0.109\\windows\\compiler\\lib\\intel64_win']
    define_macros = [('SCIPY_MKL_H', None), ('HAVE_CBLAS', None)]
    include_dirs = ['C:\\Program Files (x86)\\IntelSWTools\\parallel_studio_xe_2017.0.036\\compilers_and_libraries_2017\\windows\\mkl\\include']

C:\Python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
  warnings.warn(msg)
running config
running build_clib
running build_src
build_src
building py_modules sources
building library "npymath" sources
Could not locate executable icc
Could not locate executable ecc
customize GnuFCompiler
Could not locate executable g77
Could not locate executable f77
customize IntelVisualFCompiler
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\bin\intel64\ifort.exe
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\bin\intel64\ifort.exe
customize AbsoftFCompiler
Could not locate executable f90
customize CompaqVisualFCompiler
Could not locate executable DF
customize IntelItaniumVisualFCompiler
Could not locate executable efl
customize Gnu95FCompiler
Could not locate executable gfortran
Could not locate executable f95
customize G95FCompiler
Could not locate executable g95
customize IntelEM64VisualFCompiler
customize IntelEM64VisualFCompiler
customize IntelEM64VisualFCompiler using config
C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\bin\intel64\icl.exe /c /nologo /O3 /MD /W3 /Qstd=c99 /QxHost /fp:strict /Qopenmp -Inumpy\core\src\private -Inumpy\core\src -Inumpy\core -Inumpy\core\src\npymath -Inumpy\core\src\multiarray -Inumpy\core\src\umath -Inumpy\core\src\npysort -IC:\Python27\include -IC:\Python27\PC /Tc_configtest.c /Fo_configtest.obj
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\bin\intel64\icl.exe
xilink /nologo /INCREMENTAL:NO _configtest.obj /OUT:_configtest.exe /MANIFEST /MANIFESTFILE:_configtest.exe.manifest
Found executable C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2017.0.109\windows\bin\intel64\xilink.exe
LINK : fatal error LNK1104: cannot open file 'libiomp5md.lib'
failure.
removing: _configtest.c _configtest.obj

[Zhang_Z_Intel]

Hi,

Building NumPy with MKL on Windows has always been tricky. Many different things can go wrong, depending on the user's particular environment configuration. And the problem is difficult to debug. I'd recommend you to use a pre-built NumPy with MKL optimization. There are 2 options:

1. You can get "Intel Distribution for Python" (https://software.intel.com/en-us/intel-distribution-for-python). Many Python numeric packages, including NumPy and SciPy, are optimized and come with MKL built-in. It is a free (no-cost) download.
2. You can get the NumPy package from Anaconda, and then get Anaconda's "MKL Optimization" option (https://docs.continuum.io/mkl-optimizations/). It is also a free (no-cost) download.

[Bo1]

It seems that the Intel distribution is a whole python distribution, is it possible that confliction occurs with the original python distribution installed before? And is it possible to put the Intel built version of numpy in virtualenv of the original python distribution? Thanks.

Zhang Z. (Intel) wrote:

Hi,

Building NumPy with MKL on Windows has always been tricky. Many different things can go wrong, depending on the user's particular environment configuration. And the problem is difficult to debug. I'd recommend you to use a pre-built NumPy with MKL optimization. There are 2 options:

1. You can get "Intel Distribution for Python" (https://software.intel.com/en-us/intel-distribution-for-python). Many Python numeric packages, including NumPy and SciPy, are optimized and come with MKL built-in. It is a free (no-cost) download.
2. You can get the NumPy package from Anaconda, and then get Anaconda's "MKL Optimization" option (https://docs.continuum.io/mkl-optimizations/). It is also a free (no-cost) download.

[Robert_C_Intel]

Hi Bo,

The default install for intel python will not try to replace the system python. You have to explicitly put our python on your path. Virtualenv is not supported, but we do provide conda for managing virtual environments. If you are already using anaconda, then it is very quick to create a new environment that has intel python.

https://software.intel.com/en-us/articles/using-intel-distribution-for-python-with-anaconda

Continuum anaconda does have MKL, but intel python's numpy & scipy have some optimizations that have not landed upstream yet.

Robert

[Bo1]

Hi Robert,

Thanks for the info. I've never tried conda before (maybe it's time to do it now), nor anaconda, because I tended to install only what I needed exactly. I've installed Intel python, and I just installed virtualenv with pip in Intel python, and created a venv, it seemed to work well. When I switch venvs between this new one and previous one, different versions of python can be identified correctly. So I think maybe virtualenv can work in Intel python.

Bo

Robert C. (Intel) wrote:

Hi Bo,

The default install for intel python will not try to replace the system python. You have to explicitly put our python on your path. Virtualenv is not supported, but we do provide conda for managing virtual environments. If you are already using anaconda, then it is very quick to create a new environment that has intel python.

https://software.intel.com/en-us/articles/using-intel-distribution-for-p...

Continuum anaconda does have MKL, but intel python's numpy & scipy have some optimizations that have not landed upstream yet.

Robert