topic a problem of using Intel distribution for python in Intel® Distribution for Python*

a problem of using Intel distribution for python

Xuwen_T_Intel — Tue, 13 Jun 2017 03:50:00 GMT

Hi,

I helped someone else to post his problem here about the use of Intel distribution for Python. He used anaconda2 to install Intel distribution for python and encountered a problem when he tried to execute his code. The error messages are listed below and his problem is in CorrMM. Please provide your precious suggestions for solving this problem. His code used Theano and he installed the latest Intel distribution for python.

Thanks.

Traceback (most recent call last):

File "./deepMedicRun", line 141, in <module>

deepMedicTest.deepMedicTestMain(absPathToTestConf, absPathToCnnModelGivenInCmdLine)

File "/home/intel_ai/deepmedic/deepmedic/frontEndModules/deepMedicTest.py", line 302, in deepMedicTestMain

cnn3dInstance.compileTestAndVisualisationFunction(*testSessionParameters.getTupleForCompilationOfTestFunc())

File "/home/intel_ai/deepmedic/deepmedic/cnn3d.py", line 626, in compileTestAndVisualisationFunction

givens=givensSet

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/function.py", line 326, in function

output_keys=output_keys)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/pfunc.py", line 484, in pfunc

output_keys=output_keys)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/function_module.py", line 1786, in orig_function

defaults)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/compile/function_module.py", line 1650, in create

input_storage=input_storage_lists, storage_map=storage_map)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/link.py", line 699, in make_thunk

storage_map=storage_map)[:3]

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/vm.py", line 1051, in make_all

no_recycling))

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/op.py", line 932, in make_thunk

no_recycling)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/op.py", line 833, in make_c_thunk

output_storage=node_output_storage)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cc.py", line 1190, in make_thunk

keep_lock=keep_lock)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cc.py", line 1131, in __compile__

keep_lock=keep_lock)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cc.py", line 1589, in cthunk_factory

key=key, lnk=self, keep_lock=keep_lock)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cmodule.py", line 1145, in module_from_key

module = lnk.compile_cmodule(location)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cc.py", line 1492, in compile_cmodule

preargs=preargs)

File "/root/.local/lib/python2.7/site-packages/Theano-0.9.0.dev2-py2.7.egg/theano/gof/cmodule.py", line 2313, in compile_str

(status, compile_stderr.replace('\n', '. ')))

Exception: ('The following error happened while compiling the node', CorrMM{valid, (1, 1), (1, 1)}(Reshape{4}.0, Subtensor{::, ::, ::int64, ::int64}.0), '\n', 'Compilation failed (return status=1):

/bin/ld: cannot find -lmkl_rt. collect2: error: ld returned 1 exit status. ', '[CorrMM{valid, (1, 1), (1, 1)}(<TensorType(float32, 4D)>, <TensorType(float32, 4D)>)]')

I checked the Theano website

Xuwen_T_Intel — Tue, 13 Jun 2017 07:10:04 GMT

I checked the Theano website and found that CorrMM was carried out on the GPU in Theano 0.8. Has this API been supported by MKL in Intel Python distribution update 3 already?

CorrMM is the original CPU OP

Peng_Z_Intel — Tue, 13 Jun 2017 08:00:49 GMT

CorrMM is the original CPU OP and the Theano doesn't find the mkl_rt.so in the library path from your log.

Please add the path of mkl_rt.so into LIBRARY_PATH.

Thanks,

In case it is not clear,

Todd_T_Intel — Tue, 13 Jun 2017 14:23:43 GMT

In case it is not clear, libmkl_rt.so is found in <python_env_root>/lib in the Intel Distribution for Python.

If this does not resolve the problem, it is very helpful to post the output of "conda info" and "conda list", which tells a great deal about the environment.

-Todd

Hi, Peng and Todd

Xuwen_T_Intel — Tue, 13 Jun 2017 23:29:44 GMT

Hi, Peng and Todd

Thanks for the suggestions. We will post the output of "conda list" and "conda info" at a later time.

Anyway we ever used the "locate libmkl_rt.so" and found several libmkl_rt.so files in different directories.

find: `/run/user/1000/gvfs': Permission denied
/home/intel_ai/anaconda2/pkgs/mkl-2017.0.3-intel_6/lib/libmkl_rt.so
/home/intel_ai/anaconda2/pkgs/mkl-2017.0.1-0/lib/libmkl_rt.so
/home/intel_ai/anaconda2/envs/idp2/lib/libmkl_rt.so
/home/intel_ai/anaconda2/lib/libmkl_rt.so
/opt/intel/compilers_and_libraries_2017.2.174/Linux

Hi, Todd

Xuwen_T_Intel — Wed, 14 Jun 2017 00:11:00 GMT

Hi, Todd

The output of "conda info" and "conda list" is list below.

[intel_ai@localhost deepmedic]$ conda info

Current conda install:

platform : linux-64

conda version : 4.3.21

conda is private : False

conda-env version : 4.3.21

conda-build version : not installed

python version : 2.7.13.final.0

requests version : 2.14.2

root environment : /home/intel_ai/anaconda2 (writable)

default environment : /home/intel_ai/anaconda2

envs directories : /home/intel_ai/anaconda2/envs

/home/intel_ai/.conda/envs

package cache : /home/intel_ai/anaconda2/pkgs

/home/intel_ai/.conda/pkgs

channel URLs : https://conda.anaconda.org/intel/linux-64

https://conda.anaconda.org/intel/noarch

https://conda.anaconda.org/conda-forge/linux-64

https://conda.anaconda.org/conda-forge/noarch

https://repo.continuum.io/pkgs/free/linux-64

https://repo.continuum.io/pkgs/free/noarch

https://repo.continuum.io/pkgs/r/linux-64

https://repo.continuum.io/pkgs/r/noarch

https://repo.continuum.io/pkgs/pro/linux-64

https://repo.continuum.io/pkgs/pro/noarch

config file : /home/intel_ai/.condarc

netrc file : None

offline mode : False

user-agent : conda/4.3.21 requests/2.14.2 CPython/2.7.13 Linux/3.10.0-327.36.3.el7.xppsl_1.5.0.4036.x86_64 CentOS Linux/7.3.1611 glibc/2.17

UID:GID : 1000:1000

------------------------------------------------------------

_license 1.1 py27_1

alabaster 0.7.10 py27_0

anaconda 4.4.0 np112py27_0

anaconda-client 1.6.3 py27_0

anaconda-navigator 1.6.2 py27_0

anaconda-project 0.6.0 py27_0

asn1crypto 0.22.0 py27_0

astroid 1.4.9 py27_0

astropy 1.3.2 np112py27_0

babel 2.4.0 py27_0

backports 1.0 py27_0

backports_abc 0.5 py27_0

beautifulsoup4 4.6.0 py27_0

bitarray 0.8.1 py27_0

blaze 0.10.1 py27_0

bleach 1.5.0 py27_0

bokeh 0.12.5 py27_1

boto 2.46.1 py27_0

bottleneck 1.2.1 np112py27_0

cairo 1.14.8 0

cdecimal 2.3 py27_2

cffi 1.10.0 py27_0

chardet 3.0.3 py27_0

click 6.7 py27_0

cloudpickle 0.2.2 py27_0

clyent 1.2.2 py27_0

colorama 0.3.9 py27_0

conda 4.3.21 py27_0 conda-forge

conda-env 2.6.0 0 conda-forge

configparser 3.5.0 py27_0

contextlib2 0.5.5 py27_0

cryptography 1.8.1 py27_0

cuda80 1.0 0 soumith

curl 7.52.1 0

cycler 0.10.0 py27_0

cython 0.25.2 py27_0

cytoolz 0.8.2 py27_0

dask 0.14.3 py27_1

datashape 0.5.4 py27_0

dbus 1.10.10 0

decorator 4.0.11 py27_0

dicom 0.9.9.post1 <pip>

distributed 1.16.3 py27_0

docutils 0.13.1 py27_0

entrypoints 0.2.2 py27_1

enum34 1.1.6 py27_0

et_xmlfile 1.0.1 py27_0

expat 2.1.0 0

fastcache 1.0.2 py27_1

flask 0.12.2 py27_0

flask-cors 3.0.2 py27_0

fontconfig 2.12.1 3

freetype 2.5.5 2

funcsigs 1.0.2 py27_0

functools32 3.2.3.2 py27_0

futures 3.1.1 py27_0

get_terminal_size 1.0.0 py27_0

gevent 1.2.1 py27_0

glib 2.50.2 1

greenlet 0.4.12 py27_0

grin 1.2.1 py27_3

gst-plugins-base 1.8.0 0

gstreamer 1.8.0 0

h5py 2.7.0 np112py27_0

harfbuzz 0.9.39 2

hdf5 1.8.17 1

heapdict 1.0.0 py27_1

html5lib 0.999 py27_0

icu 54.1 0

idna 2.5 py27_0

imagesize 0.7.1 py27_0

ipaddress 1.0.18 py27_0

ipykernel 4.6.1 py27_0

ipython 5.3.0 py27_0

ipython_genutils 0.2.0 py27_0

ipywidgets 6.0.0 py27_0

isort 4.2.5 py27_0

itsdangerous 0.24 py27_0

jbig 2.1 0

jdcal 1.3 py27_0

jedi 0.10.2 py27_2

jinja2 2.9.6 py27_0

jpeg 9b 0

jsonschema 2.6.0 py27_0

jupyter 1.0.0 py27_3

jupyter_client 5.0.1 py27_0

jupyter_console 5.1.0 py27_0

jupyter_core 4.3.0 py27_0

lazy-object-proxy 1.2.2 py27_0

libffi 3.2.1 1

libgcc 4.8.5 2

libgfortran 3.0.0 1

libiconv 1.14 0

libpng 1.6.27 0

libprotobuf 3.2.0 0 anaconda

libsodium 1.0.10 0

libtiff 4.0.6 3

libtool 2.4.2 0

libxcb 1.12 1

libxml2 2.9.4 0

libxslt 1.1.29 0

llvmlite 0.18.0 py27_0

locket 0.2.0 py27_1

lxml 3.7.3 py27_0

markupsafe 0.23 py27_2

matplotlib 2.0.2 np112py27_0

mistune 0.7.4 py27_0

mkl 2017.0.1 0

mkl-service 1.1.2 py27_3

mock 2.0.0 py27_0 anaconda

mpmath 0.19 py27_1

msgpack-python 0.4.8 py27_0

multipledispatch 0.4.9 py27_0

navigator-updater 0.1.0 py27_0

nbconvert 5.1.1 py27_0

nbformat 4.3.0 py27_0

networkx 1.11 py27_0

nibabel 2.1.0 py27_0 conda-forge

nltk 3.2.3 py27_0

nose 1.3.7 py27_1

notebook 5.0.0 py27_0

numba 0.33.0 np112py27_0

numexpr 2.6.2 np112py27_0

numpy 1.12.1 py27_0

numpydoc 0.6.0 py27_0

odo 0.5.0 py27_1

olefile 0.44 py27_0

openpyxl 2.4.7 py27_0

openssl 1.0.2l 0

packaging 16.8 py27_0

pandas 0.20.1 np112py27_0

pandocfilters 1.4.1 py27_0

pango 1.40.3 1

partd 0.3.8 py27_0

path.py 10.3.1 py27_0

pathlib2 2.2.1 py27_0

patsy 0.4.1 py27_0

pbr 1.10.0 py27_0 anaconda

pcre 8.39 1

pep8 1.7.0 py27_0

pexpect 4.2.1 py27_0

pickleshare 0.7.4 py27_0

pillow 4.1.1 py27_0

pip 9.0.1 py27_1

pixman 0.34.0 0

ply 3.10 py27_0

pp 1.6.4 py27_0 geneko

prompt_toolkit 1.0.14 py27_0

protobuf 3.2.0 py27_0 anaconda

psutil 5.2.2 py27_0

ptyprocess 0.5.1 py27_0

py 1.4.33 py27_0

pycairo 1.10.0 py27_0

pycosat 0.6.2 py27_0

pycparser 2.17 py27_0

pycrypto 2.6.1 py27_6

pycurl 7.43.0 py27_2

pydicom 0.9.9 <pip>

pydicom 0.9.9 py27_0 conda-forge

pyflakes 1.5.0 py27_0

pygments 2.2.0 py27_0

pylint 1.6.4 py27_1

pyodbc 4.0.16 py27_0

pyopenssl 17.0.0 py27_0

pyparsing 2.1.4 py27_0

pyqt 5.6.0 py27_2

pytables 3.3.0 np112py27_0

pytest 3.0.7 py27_0

python 2.7.13 0

python-dateutil 2.6.0 py27_0

pytorch 0.1.12 py27_2cu80 [cuda80] soumith

pytz 2017.2 py27_0

pywavelets 0.5.2 np112py27_0

pyyaml 3.12 py27_0

pyzmq 16.0.2 py27_0

qt 5.6.2 4

qtawesome 0.4.4 py27_0

qtconsole 4.3.0 py27_0

qtpy 1.2.1 py27_0

readline 6.2 2

requests 2.14.2 py27_0

rope 0.9.4 py27_1

ruamel_yaml 0.11.14 py27_1

scandir 1.5 py27_0

scikit-image 0.13.0 np112py27_0

scikit-learn 0.18.1 np112py27_1

scipy 0.19.0 np112py27_0

seaborn 0.7.1 py27_0

setuptools 27.2.0 py27_0

simplegeneric 0.8.1 py27_1

singledispatch 3.4.0.3 py27_0

sip 4.18 py27_0

six 1.10.0 py27_0

snowballstemmer 1.2.1 py27_0

sortedcollections 0.5.3 py27_0

sortedcontainers 1.5.7 py27_0

sphinx 1.5.6 py27_0

spyder 3.1.4 py27_0

sqlalchemy 1.1.9 py27_0

sqlite 3.13.0 0

ssl_match_hostname 3.4.0.2 py27_1

statsmodels 0.8.0 np112py27_0

subprocess32 3.2.7 py27_0

sympy 1.0 py27_0

tblib 1.3.2 py27_0

tensorflow 1.1.0 np112py27_0 anaconda

terminado 0.6 py27_0

testpath 0.3 py27_0

tk 8.5.18 0

toolz 0.8.2 py27_0

torchvision 0.1.8 py27_2 soumith

tornado 4.5.1 py27_0

traitlets 4.3.2 py27_0

unicodecsv 0.14.1 py27_0

unixodbc 2.3.4 0

wcwidth 0.1.7 py27_0

werkzeug 0.12.2 py27_0

wheel 0.29.0 py27_0

widgetsnbextension 2.0.0 py27_0

wrapt 1.10.10 py27_0

xlrd 1.0.0 py27_0

xlsxwriter 0.9.6 py27_0

xlwt 1.2.0 py27_0

xz 5.2.2 1

yaml 0.1.6 0

zeromq 4.1.5 0

zict 0.1.2 py27_0

zlib 1.2.8 3