oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

Nathan_Peper — Tue, 03 May 2022 16:08:47 GMT

I have a NUC with a fresh Ubuntu 20.04 LTS install that I’m trying to install the Intel oneAPI AI Analytics Toolkit on. I can get everything to work through the GUI based installer and if I use APT, but if I only install Miniconda and then use the Conda Package Manager I run into some issues. I’m not sure if there is an undocumented step(s) or prereq(s) that I’m missing. There’s a comment about setting up the environment variables to access the tools, but it doesn’t look like I can use the “source /opt/intel/oneapi/setvars.sh” like in the other install methods when only using conda and no previous oneAPI installs through the other methods.

I’m following the instructions as outlined in the below link on our site, but get the below results in the terminal. I run into this error when I try to import tensorflow in python.

Thanks for the help,

Nate

Snapshot of the error:

(oneapi-tf) npeper@TGL-i7-NUC-1:~$ python -c "import tensorflow as tf"

Traceback (most recent call last):

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/__init__.py", line 22, in <module>

from . import multiarray

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/multiarray.py", line 12, in <module>

from . import overrides

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/overrides.py", line 7, in <module>

from numpy.core._multiarray_umath import (

ImportError: libmkl_rt.so.1: cannot open shared object file: No such file or directory

Install instructions on our website:

https://www.intel.com/content/www/us/en/develop/documentation/installation-guide-for-intel-oneapi-toolkits-linux/top/installation/install-using-package-managers/conda/install-intel-ai-analytics-toolkit-via-conda.html

Full terminal steps and output:

npeper@TGL-i7-NUC-1:~$ source /home/npeper/miniconda3/bin/activate

(base) npeper@TGL-i7-NUC-1:~$ conda create -n oneapi-tf -c intel intel-aikit-tensorflow

Collecting package metadata (current_repodata.json): done

Solving environment: done

## Package Plan ##

environment location: /home/npeper/miniconda3/envs/oneapi-tf

added / updated specs:

- intel-aikit-tensorflow

The following NEW packages will be INSTALLED:

absl-py intel/linux-64::absl-py-0.12.0-py37h06a4308_0

aiohttp intel/linux-64::aiohttp-3.7.4-py37h27cfd23_1

asn1crypto intel/linux-64::asn1crypto-1.4.0-py37hcd5400e_2

astor intel/linux-64::astor-0.8.1-py37h06a4308_0

astunparse intel/noarch::astunparse-1.6.3-py_0

async-timeout intel/linux-64::async-timeout-3.0.1-py37h06a4308_0

attrs intel/noarch::attrs-21.2.0-pyhd3eb1b0_0

blinker intel/linux-64::blinker-1.4-py37h06a4308_0

bzip2 intel/linux-64::bzip2-1.0.8-hb9a14ef_9

c-ares intel/linux-64::c-ares-1.17.1-h14c3975_3

ca-certificates intel/linux-64::ca-certificates-2021.10.26-h06a4308_2

cached-property intel/noarch::cached-property-1.5.2-py_0

cachetools intel/noarch::cachetools-4.2.2-pyhd3eb1b0_0

certifi intel/linux-64::certifi-2021.10.8-py37h06a4308_0

cffi intel/linux-64::cffi-1.14.6-py37h400218f_0

chardet intel/linux-64::chardet-3.0.4-py37h82fad40_4

click intel/noarch::click-8.0.3-pyhd3eb1b0_0

cloudpickle intel/noarch::cloudpickle-1.6.0-py_0

contextlib2 intel/noarch::contextlib2-0.5.5-py_2

coverage intel/linux-64::coverage-5.5-py37h27cfd23_2

cryptography intel/linux-64::cryptography-3.3.2-py37h3d3d174_1

cython intel/linux-64::cython-0.29.24-py37hbf8da92_2

daal4py intel/linux-64::daal4py-2021.5.1-py37_intel_803

dal intel/linux-64::dal-2021.5.1-intel_803

dataclasses intel/noarch::dataclasses-0.8-pyh6d0b6a4_7

dpcpp-cpp-rt intel/linux-64::dpcpp-cpp-rt-2022.0.1-intel_3633

dpcpp_cpp_rt intel/linux-64::dpcpp_cpp_rt-2022.0.1-intel_3633

flatbuffers intel/linux-64::flatbuffers-2.0.0-h2531618_0

fortran_rt intel/linux-64::fortran_rt-2022.0.1-intel_3633

future intel/linux-64::future-0.18.2-py37_1

gast intel/noarch::gast-0.4.0-py_0

google-auth intel/noarch::google-auth-1.33.0-pyhd3eb1b0_0

google-auth-oauth~ intel/noarch::google-auth-oauthlib-0.4.1-py_2

google-pasta intel/noarch::google-pasta-0.2.0-py_0

grpcio intel/linux-64::grpcio-1.36.1-py37h2157cd5_1

h5py intel/linux-64::h5py-3.2.1-py37h6c542dc_0

hdf5 intel/linux-64::hdf5-1.10.6-hb1b8bf9_0

hyperopt intel/noarch::hyperopt-0.2.5-pyh9f0ad1d_0

icc_rt intel/linux-64::icc_rt-2022.0.1-intel_3633

idna intel/linux-64::idna-3.1-py37hf52dec2_0

impi-devel intel/linux-64::impi-devel-2021.5.0-intel_497

impi_rt intel/linux-64::impi_rt-2021.5.0-intel_pycpuinfo_496

importlib-metadata intel/linux-64::importlib-metadata-3.10.0-py37h06a4308_0

intel-aikit-tenso~ intel/linux-64::intel-aikit-tensorflow-2021.3.0-0

intel-cmplr-lib-rt intel/linux-64::intel-cmplr-lib-rt-2022.0.1-intel_3633

intel-cmplr-lic-rt intel/linux-64::intel-cmplr-lic-rt-2022.0.1-intel_3633

intel-fortran-rt intel/linux-64::intel-fortran-rt-2022.0.1-intel_3633

intel-opencl-rt intel/linux-64::intel-opencl-rt-2022.0.1-intel_3633

intel-openmp intel/linux-64::intel-openmp-2022.0.1-intel_3633

intelpython intel/linux-64::intelpython-2022.0.0-0

joblib intel/linux-64::joblib-1.0.1-py37h3f38642_0

keras-preprocessi~ intel/noarch::keras-preprocessing-1.1.2-pyhd3eb1b0_0

libffi intel/linux-64::libffi-3.3-14

libgcc-ng intel/linux-64::libgcc-ng-9.3.0-hdf63c60_101

libgfortran-ng intel/linux-64::libgfortran-ng-7.3.0-hdf63c60_0

libprotobuf intel/linux-64::libprotobuf-3.14.0-he78db7b_1

libstdcxx-ng intel/linux-64::libstdcxx-ng-9.3.0-hdf63c60_101

lpot intel/noarch::lpot-1.0-py_0

markdown intel/linux-64::markdown-3.3.4-py37h06a4308_0

mkl intel/linux-64::mkl-2022.0.1-intel_117

mkl-service intel/linux-64::mkl-service-2.4.0-py37h4119f30_11

mkl_fft intel/linux-64::mkl_fft-1.2.0-py37hb9c2cde_4

mkl_random intel/linux-64::mkl_random-1.2.0-py37h90a4e38_4

mkl_umath intel/linux-64::mkl_umath-0.1.0-py37hb46b53c_0

multidict intel/linux-64::multidict-5.1.0-py37h27cfd23_2

networkx intel/noarch::networkx-2.6.2-pyhd3eb1b0_2

numpy intel/linux-64::numpy-1.19.2-py37h02626c5_0

numpy-base intel/linux-64::numpy-base-1.19.2-py37h591eb60_0

oauthlib intel/noarch::oauthlib-3.1.1-pyhd3eb1b0_0

openssl intel/linux-64::openssl-1.1.1l-h7f8727e_0

opt_einsum intel/noarch::opt_einsum-3.3.0-pyhd3eb1b0_1

pandas intel/linux-64::pandas-1.2.0-py37hef31ef9_2

pip intel/linux-64::pip-21.2.2-py37h06a4308_0

protobuf intel/linux-64::protobuf-3.14.0-py37h2531618_1

py-cpuinfo intel/noarch::py-cpuinfo-8.0.0-pyhd3eb1b0_0

pyasn1 intel/noarch::pyasn1-0.4.8-py_0

pyasn1-modules intel/noarch::pyasn1-modules-0.2.8-py_0

pycparser intel/noarch::pycparser-2.21-pyhd3eb1b0_0

pyjwt intel/noarch::pyjwt-2.1.0-pyhd8ed1ab_0

pymongo intel/linux-64::pymongo-3.11.4-py37h2531618_0

pyopenssl intel/noarch::pyopenssl-21.0.0-pyhd3eb1b0_1

pysocks intel/linux-64::pysocks-1.7.1-py37_1

python intel/linux-64::python-3.7.11-h6ca7d21_6

python-dateutil intel/linux-64::python-dateutil-2.8.2-py37_1

pytz intel/noarch::pytz-2021.3-pyhd3eb1b0_0

pyyaml intel/linux-64::pyyaml-6.0-py37h7f8727e_1

requests intel/linux-64::requests-2.25.1-py37h6282448_1

requests-oauthlib intel/noarch::requests-oauthlib-1.3.0-py_0

rsa intel/noarch::rsa-4.7.2-pyhd3eb1b0_1

schema intel/noarch::schema-0.7.4-pyh44b312d_0

scikit-learn intel/linux-64::scikit-learn-0.24.2-py37h8411759_4

scikit-learn-inte~ intel/linux-64::scikit-learn-intelex-2021.5.1-py37_intel_803

scipy intel/linux-64::scipy-1.5.2-py37h83b3cbf_0

setuptools intel/linux-64::setuptools-58.0.4-py37h06a4308_0

six intel/linux-64::six-1.16.0-py37_0

sqlite intel/linux-64::sqlite-3.36.0-hb9a14ef_3

tbb intel/linux-64::tbb-2021.5.0-intel_707

tbb4py intel/linux-64::tbb4py-2021.5.0-py37_intel_707

tensorboard intel/noarch::tensorboard-2.5.0-py_0

tensorboard-plugi~ intel/noarch::tensorboard-plugin-wit-1.6.0-py_0

tensorflow intel/linux-64::tensorflow-2.5.0-py37_1

tensorflow-estima~ intel/noarch::tensorflow-estimator-2.5.0-pyh7b7c402_0

termcolor intel/linux-64::termcolor-1.1.0-py37h06a4308_1

threadpoolctl intel/noarch::threadpoolctl-2.2.0-pyh0d69192_0

tk intel/linux-64::tk-8.6.10-h8e2d9d6_3

tqdm intel/noarch::tqdm-4.62.3-pyhd3eb1b0_1

typing-extensions intel/noarch::typing-extensions-3.7.4.3-hd3eb1b0_0

typing_extensions intel/noarch::typing_extensions-3.7.4.3-pyh06a4308_0

urllib3 intel/linux-64::urllib3-1.26.6-py37h6fed32d_1

werkzeug intel/noarch::werkzeug-2.0.2-pyhd3eb1b0_0

wheel intel/noarch::wheel-0.37.0-pyhd3eb1b0_1

wrapt intel/linux-64::wrapt-1.12.1-py37h7b6447c_1

xz intel/linux-64::xz-5.2.5-h74280d8_2

yaml intel/linux-64::yaml-0.2.5-h7b6447c_0

yarl intel/linux-64::yarl-1.6.3-py37h27cfd23_0

zipp intel/noarch::zipp-3.6.0-pyhd3eb1b0_0

zlib intel/linux-64::zlib-1.2.11.1-h1e99aa7_5

Proceed ([y]/n)? y

Preparing transaction: done

Verifying transaction: done

Executing transaction: /

Installed package of scikit-learn can be accelerated using scikit-learn-intelex.

More details are available here: https://intel.github.io/scikit-learn-intelex

For example:

$ conda install scikit-learn-intelex

$ python -m sklearnex my_application.py

done

# To activate this environment, use

# $ conda activate oneapi-tf

# To deactivate an active environment, use

# $ conda deactivate

(base) npeper@TGL-i7-NUC-1:~$ conda activate oneapi-tf

(oneapi-tf) npeper@TGL-i7-NUC-1:~$ python -c "import tensorflow as tf"

Traceback (most recent call last):

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/__init__.py", line 22, in <module>

from . import multiarray

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/multiarray.py", line 12, in <module>

from . import overrides

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/overrides.py", line 7, in <module>

from numpy.core._multiarray_umath import (

ImportError: libmkl_rt.so.1: cannot open shared object file: No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

File "<string>", line 1, in <module>

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/tensorflow/__init__.py", line 41, in <module>

from tensorflow.python.tools import module_util as _module_util

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/tensorflow/python/__init__.py", line 40, in <module>

from tensorflow.python.eager import context

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/tensorflow/python/eager/context.py", line 29, in <module>

import numpy as np

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/__init__.py", line 156, in <module>

from . import core

File "/home/npeper/miniconda3/envs/oneapi-tf/lib/python3.7/site-packages/numpy/core/__init__.py", line 48, in <module>

raise ImportError(msg)

ImportError:

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for

many reasons, often due to issues with your setup or how NumPy was

installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

* The Python version is: Python3.7 from "/home/npeper/miniconda3/envs/oneapi-tf/bin/python"

* The NumPy version is: "1.19.2"

and make sure that they are the versions you expect.

Please carefully study the documentation linked above for further help.

Original error was: libmkl_rt.so.1: cannot open shared object file: No such file or directory

(oneapi-tf) npeper@TGL-i7-NUC-1:~$ cd /opt

(oneapi-tf) npeper@TGL-i7-NUC-1:/opt$ ls

google

Re:oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

VaradJ_Intel — Thu, 05 May 2022 12:36:04 GMT

Hi,

Thank you for posting in Intel communities.

We could reproduce your issue and we have a workaround for that. Try locating the libmkl_rt.so with the below command:

locate libmkl_rt.so

If locate command is not found, try installing with the following command:

sudo apt install mlocate

Try renaming all the files to libmkl_rt.so.1 with the below command:

mv <path>/libmkl_rt.so <path>/libmkl_rt.so.1

You would be getting an error that flatbuffers module is not found. You could resolve it by simply installing that package with the below command:

conda install -c conda-forge python-flatbuffers

Try importing tensorflow again & let us know if the issue still persists.

Thank You.

Re: oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

Nathan_Peper — Mon, 09 May 2022 04:38:07 GMT

Thank you so much for the help on this, it looks like that method works for the tensorflow import. While continuing to troubleshoot, I also found this as a fix to the issue:

https://stackoverflow.com/questions/58868528/importing-the-numpy-c-extensions-failed

pip uninstall -y numpy
pip uninstall -y setuptools
pip install setuptools
pip install numpy
conda install -c conda-forge python-flatbuffers

Is one fix more robust or proper than the other here when looking to have all of the oneAPI features enabled?

Thank you!

topic oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS in AI Tools from Intel

oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

Re:oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

Re: oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS

Re:oneAPI AI Anlytics Toolkit Install via Conda Package Manager on Clean Ubuntu 20.04 LTS