Cannot link against MKL installed from pip

[Cortes__David]

In Linux(Ubuntu and OpenSuSe), I've tried installing MKL through pip:
 

python3 -m pip install mkl-devel --user

sudo pip3 install mkl-devel

Which will install the headers under

$HOME/.local/include

 and shared objects under

$HOME/.local/lib

But then I try compiling some file against it using gcc or clang with these arguments:

g++ the_file.cpp I$HOME/.local/include -l:$HOME/.local/lib/libmkl_rt.so

And get the following error:

/usr/bin/ld: cannot find -l:/home/david/.local/libmkl_rt.so

Even though the file exists and I've set all permissions for everyone:

ls -s /home/david/.local/libmkl_rt.so

Shows these permissions:

-rwxrwxrwx 1

[Aaron_J_Intel2 · Accepted solution]

Hi David,

The path and the particular library to link are separate GNU compiler parameters.  Try the following changes,

g++ the_file.cpp -I$HOME/.local/include -L $HOME/.local/lib/ -lmkl_rt.so

And when the program runs, set an environment variable.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/lib/ ./a.out

[Cortes__David]

Thanks, that worked. Any hint on why passing the file with -l: would work for user and system-wide installs from e.g. conda or package manager, whereas with a pip install it wouldn’t?

[Aaron_J_Intel2]

I am most familiar with using 

python -m virtualenv

applications and the way those work is by modifying env variables which when activated change the PATH et al.  I think this might be similiar to how conda is implemented.  There is probably some setup script provided by the package manager and conda that the pip package is missing.

I'm not familiar with the "-l:" usage.  I have not found any documentation about how it works.  In particular the colon aspect, I have not seen that before.

First I would verify if the path for g++ and ld are the same in all cases.

which g++

which ld

And also check versions in all cases; pip, conda, package manager.

ld -v

g++ --version

And next, look at the environment variables set for each shell.   In particular PATH, LD_LIBRARY_PATH, LD_PRELOAD, and env variables with CONDA_ prefix.  Read about how those influence each setup.

NB: In my experience it can be easy to get into an environment that uses a mixture of solutions when learning the differences between ways to do this, so while deciphering be careful to not install several solutions on the same machine simultaneously.  It is entirely possible there is overlap, unless only one way is done on each clean installation.

[Gennady_F_Intel]

I think you could try to ask the python related questions to the IDP Forum - https://software.intel.com/en-us/forums/intel-distribution-for-python

[Cortes__David]

I didn't manage to figure out why a pip install wouldn't end up allowing more compilation options (setting similar environment variables didn't make a difference), but in the end found that with the pip install, it's possible to compile like this too:

g++ the_file.cpp -I$HOME/.local/include -L$HOME/.local/lib -l:libmkl_rt.so -Wl,-rpath=$HOME/.local/lib

Then it will work fine.