Cannot find BLAS on a machine with MKL when installing scipy via pip

[Ziyuan]

I installed Intel MKL and other libraries for a customized numpy. Here is my `~/.numpy-site.cfg`:

    [DEFAULT]
    library_dirs = /usr/lib:/usr/local/lib
    include_dirs = /usr/include:/usr/local/include
    
    [mkl]
    library_dirs = /opt/intel/mkl/lib/intel64/
    include_dirs = /opt/intel/mkl/include/
    mkl_libs = mkl_intel_ilp64, mkl_intel_thread, mkl_core, mkl_rt
    lapack_libs =
    
    [amd]
    amd_libs = amd
    
    [umfpack]
    umfpack_libs = umfpack
    
    [djbfft]
    include_dirs = /usr/local/djbfft/include
    library_dirs = /usr/local/djbfft/lib

This configuration file seems OK during the installation of numpy. But when I was installing scipy via `pip3 install scipy`, it reported that

    numpy.distutils.system_info.BlasNotFoundError:
    
        Blas (http://www.netlib.org/blas/) libraries not found.
    
        Directories to search for the libraries can be specified in the
    
        numpy/distutils/site.cfg file (section [blas]) or by setting
    
        the BLAS environment variable.

In my mind MKL is an implementation of Blas so just mentioning MKL should be fine. I've tried

 1. `export LD_LIBRARY_PATH=/opt/intel/mkl/lib/intel64:$LD_LIBRARY_PATH‌​`
 2. `export BLAS=/opt/intel/mkl/lib/intel64`
 3. Copy the content in the `[mkl]` section and paste into the `[blas]` section in the file `~/.numpy-site.cfg`

But none of these works. So what is going wrong? Does scipy respect `~/.numpy-site.cfg`? Thank you.

[mecej4]

I don't have an answer specific to scipy, but when you port any configure/make scripts from Gcc/GFortran/Netlib BLAS/Lapack to Ifort/MKL, you may need to adapt the scripts in view of the fact that MKL includes BLAS as well as Lapack, so that it suffices to link against MKL. 

[Ziyuan]

mecej4 wrote:

I don't have an answer specific to scipy, but when you port any configure/make scripts from Gcc/GFortran/Netlib BLAS/Lapack to Ifort/MKL, you may need to adapt the scripts in view of the fact that MKL includes BLAS as well as Lapack, so that it suffices to link against MKL. 

Yes I think so too. Scipy does support MKL, and this is an example about how to write the ~/.numpy-site.cfg: https://github.com/numpy/numpy/blob/master/site.cfg.example

[VipinKumar_E_Intel]

Please refer this article on installing scipy/numpy with mkl

http://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl

--Vipin

[Ziyuan]

Vipin Kumar E K (Intel) wrote:

Please refer this article on installing scipy/numpy with mkl

http://software.intel.com/en-us/articles/numpyscipy-with-intel-mkl

--Vipin

It's for installing from source, while I am asking about installing via pip.

[VipinKumar_E_Intel]

Can you modify your site.config for mkl_libs instead of

mkl_libs = mkl_intel_ilp64, mkl_intel_thread, mkl_core, mkl_rt

to

mkl_libs =  mkl_rt

mkl_rt is the runtime dynamic lib and you don't need to specify other libs from different layers.

Also, set the MKl env variables by running

$source /opt/intel/mkl/bin/mklvars.sh ?

Vipin

[Ziyuan]

Vipin Kumar E K (Intel) wrote:

Can you modify your site.config for mkl_libs instead of

mkl_libs = mkl_intel_ilp64, mkl_intel_thread, mkl_core, mkl_rt

to

mkl_libs =  mkl_rt

mkl_rt is the runtime dynamic lib and you don't need to specify other libs from different layers.

Also, set the MKl env variables by running

$source /opt/intel/mkl/bin/mklvars.sh ?

Vipin

Hi. It doesn't work either.

$ source /opt/intel/mkl/bin/mklvars.sh intel64 ilp64
$ echo $MKLROOT
/opt/intel/composer_xe_2013_sp1.2.144/mkl
$ sudo pip3 install scipy

will give the same error. Particularly, here are the messages about not finding the libs:

Installing collected packages: scipy
  Running setup.py install for scipy
    blas_opt_info:
    blas_mkl_info:
      libraries mkl,vml,guide not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    openblas_info:
      libraries  not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    atlas_blas_threads_info:
    Setting PTATLAS=ATLAS
      libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    atlas_blas_info:
      libraries f77blas,cblas,atlas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    blas_info:
      libraries blas not found in ['/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
      NOT AVAILABLE
    
    blas_src_info:
      NOT AVAILABLE
    
    /usr/local/lib/python3.3/dist-packages/numpy/distutils/system_info.py:1521: UserWarning:
        Atlas (http://math-atlas.sourceforge.net/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [atlas]) or by setting
        the ATLAS environment variable.
      warnings.warn(AtlasNotFoundError.__doc__)
    /usr/local/lib/python3.3/dist-packages/numpy/distutils/system_info.py:1530: UserWarning:
        Blas (http://www.netlib.org/blas/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [blas]) or by setting
        the BLAS environment variable.
      warnings.warn(BlasNotFoundError.__doc__)
    /usr/local/lib/python3.3/dist-packages/numpy/distutils/system_info.py:1533: UserWarning:
        Blas (http://www.netlib.org/blas/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [blas_src]) or by setting
        the BLAS_SRC environment variable.

It seems that the MKL paths haven't gone into the environmental variables.

[Ying_H_Intel]

Hi ziyuan, 

Could you please try the Root to install the scipy? 

I haven't investigated how pip3 works, but sudo can't inherit the environment setting from  source /opt/intel/mkl/bin/mklvars.sh intel64 ilp64

as we mentioned in 

https://software.intel.com/en-us/forums/topic/277619#comment-1794098. ;

Best Regards,

Ying