Community
cancel
Showing results for 
Search instead for 
Did you mean: 
A__Adham
Beginner
455 Views

Running Unity MLAgents Error

I installed the distribution using the command: conda create -n idp -c intel python=3.6 intelpython3_full, and everything seems working.

But when I tried to train using unity mlagent using the regular command mlagents-learn config/config.yaml --run-id=project1 --train I receive the following error message : -

 

ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
ModuleNotFoundError: No module named 'numpy.core._multiarray_umath'
Traceback (most recent call last):
  File "/anaconda3/envs/idp/bin/mlagents-learn", line 11, in <module>
    load_entry_point('mlagents', 'console_scripts', 'mlagents-learn')()
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 480, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2693, in load_entry_point
    return ep.load()
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in load
    return self.resolve()
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2330, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/adam/ml-agents/ml-agents/mlagents/trainers/__init__.py", line 4, in <module>
    from .models import *
  File "/Users/adam/ml-agents/ml-agents/mlagents/trainers/models.py", line 4, in <module>
    import tensorflow as tf
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/__init__.py", line 24, in <module>
    from tensorflow.python import *  # pylint: disable=redefined-builtin
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 82, in <module>
    from tensorflow.python.estimator import estimator_lib as estimator
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/python/estimator/estimator_lib.py", line 37, in <module>
    from tensorflow.python.estimator.inputs import inputs
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/python/estimator/inputs/inputs.py", line 22, in <module>
    from tensorflow.python.estimator.inputs.numpy_io import numpy_input_fn
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/python/estimator/inputs/numpy_io.py", line 26, in <module>
    from tensorflow.python.estimator.inputs.queues import feeding_functions
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/tensorflow/python/estimator/inputs/queues/feeding_functions.py", line 40, in <module>
    import pandas as pd
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pandas/__init__.py", line 26, in <module>
    from pandas._libs import (hashtable as _hashtable,
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pandas/_libs/__init__.py", line 4, in <module>
    from .tslibs import (
  File "/anaconda3/envs/idp/lib/python3.6/site-packages/pandas/_libs/tslibs/__init__.py", line 4, in <module>
    from .conversion import normalize_date, localize_pydatetime, tz_convert_single
  File "__init__.pxd", line 918, in init pandas._libs.tslibs.conversion
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject

 

0 Kudos
5 Replies
David_L_Intel1
Employee
455 Views

Can you list your conda list and conda info?  It is strange that it cannot find core._multiarray_umath as it was introduced in NumPy 1.16.0 (and our latest is 1.16.1).  

A__Adham
Beginner
455 Views

my numpy version is 1.14.5

Python 3.6.8 |Intel Corporation| (default, Feb 27 2019, 19:23:40) 
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import numpy
>>> numpy.__version__
'1.14.5'

Conda list:

# packages in environment at /anaconda3/envs/idp:
#
# Name                    Version                   Build  Channel
absl-py                   0.7.1                    pypi_0    pypi
appnope                   0.1.0                    pypi_0    pypi
asn1crypto                0.24.0                   py36_3    intel
astor                     0.7.1                    pypi_0    pypi
atomicwrites              1.3.0                    pypi_0    pypi
attrs                     19.1.0                   pypi_0    pypi
backcall                  0.1.0                    pypi_0    pypi
bleach                    1.5.0                    pypi_0    pypi
bzip2                     1.0.6                        17    intel
certifi                   2018.1.18                py36_2    intel
cffi                      1.11.5                   py36_3    intel
chardet                   3.0.4                    py36_3    intel
cryptography              2.3                      py36_1    intel
cycler                    0.10.0                   py36_7    intel
cython                    0.29.3                   py36_1    intel
daal                      2019.3                intel_199    intel
daal4py                   2019.3           py36h7b7c402_6    intel
decorator                 4.4.0                    pypi_0    pypi
defusedxml                0.5.0                    pypi_0    pypi
docopt                    0.6.2                    pypi_0    pypi
entrypoints               0.3                      pypi_0    pypi
freetype                  2.9                           3    intel
funcsigs                  1.0.2                    py36_7    intel
gast                      0.2.2                    pypi_0    pypi
grpcio                    1.11.1                   pypi_0    pypi
html5lib                  0.9999999                pypi_0    pypi
icc_rt                    2019.3                intel_199    intel
idna                      2.6                      py36_3    intel
intel-openmp              2019.3                intel_199    intel
intelpython               2019.3                        0    intel
intelpython3_core         2019.3                        0    intel
intelpython3_full         2019.3                        0    intel
ipykernel                 5.1.0                    pypi_0    pypi
ipython                   7.4.0                    pypi_0    pypi
ipython-genutils          0.2.0                    pypi_0    pypi
ipywidgets                7.4.2                    pypi_0    pypi
jedi                      0.13.3                   pypi_0    pypi
jinja2                    2.10.1                   pypi_0    pypi
jsonschema                3.0.1                    pypi_0    pypi
jupyter                   1.0.0                    pypi_0    pypi
jupyter-client            5.2.4                    pypi_0    pypi
jupyter-console           6.0.0                    pypi_0    pypi
jupyter-core              4.4.0                    pypi_0    pypi
kiwisolver                1.0.1                    py36_2    intel
libffi                    3.2.1                        11    intel
libpng                    1.6.36                        2    intel
llvmlite                  0.27.1                   py36_0    intel
markdown                  3.1                      pypi_0    pypi
markupsafe                1.1.1                    pypi_0    pypi
matplotlib                3.0.1                    py36_1    intel
mistune                   0.8.4                    pypi_0    pypi
mkl                       2019.3                intel_199    intel
mkl-service               1.0.0                    py36_7    intel
mkl_fft                   1.0.11           py36h7b7c402_0    intel
mkl_random                1.0.2            py36h7b7c402_4    intel
mlagents                  0.8.1                     dev_0    <develop>
more-itertools            7.0.0                    pypi_0    pypi
nbconvert                 5.4.1                    pypi_0    pypi
nbformat                  4.4.0                    pypi_0    pypi
notebook                  5.7.8                    pypi_0    pypi
numba                     0.42.1              np116py36_0    intel
numexpr                   2.6.8                    py36_2    intel
numpy                     1.14.5                   pypi_0    pypi
openssl                   1.0.2r                        0    intel
pandas                    0.24.1                   py36_3    intel
pandocfilters             1.4.2                    pypi_0    pypi
parso                     0.4.0                    pypi_0    pypi
pexpect                   4.7.0                    pypi_0    pypi
pickleshare               0.7.5                    pypi_0    pypi
pillow                    6.0.0                    pypi_0    pypi
pip                       19.0.3                   pypi_0    pypi
pluggy                    0.9.0                    pypi_0    pypi
prometheus-client         0.6.0                    pypi_0    pypi
prompt-toolkit            2.0.9                    pypi_0    pypi
protobuf                  3.6.1                    pypi_0    pypi
ptyprocess                0.6.0                    pypi_0    pypi
py                        1.8.0                    pypi_0    pypi
pycosat                   0.6.3                    py36_3    intel
pycparser                 2.18                     py36_2    intel
pygments                  2.3.1                    pypi_0    pypi
pyopenssl                 17.5.0                   py36_2    intel
pyparsing                 2.2.0                    py36_2    intel
pyrsistent                0.14.11                  pypi_0    pypi
pysocks                   1.6.7                    py36_1    intel
pytest                    3.10.1                   pypi_0    pypi
python                    3.6.8                         6    intel
python-dateutil           2.6.0                   py36_12    intel
pytz                      2018.4                   py36_3    intel
pyyaml                    4.1                      py36_3    intel
pyzmq                     18.0.1                   pypi_0    pypi
qtconsole                 4.4.3                    pypi_0    pypi
requests                  2.20.1                   py36_1    intel
ruamel_yaml               0.11.14                  py36_4    intel
scikit-learn              0.20.2           py36h7b7c402_2    intel
scipy                     1.2.0                    py36_3    intel
send2trash                1.5.0                    pypi_0    pypi
setuptools                39.0.1                   py36_0    intel
six                       1.11.0                   py36_3    intel
sqlite                    3.27.2                        2    intel
tbb                       2019.4                intel_199    intel
tbb4py                    2019.4             py36_intel_0    intel
tcl                       8.6.4                        22    intel
tensorboard               1.7.0                    pypi_0    pypi
tensorflow                1.7.1                    pypi_0    pypi
termcolor                 1.1.0                    pypi_0    pypi
terminado                 0.8.2                    pypi_0    pypi
testpath                  0.4.2                    pypi_0    pypi
tk                        8.6.4                        28    intel
tornado                   6.0.2                    pypi_0    pypi
traitlets                 4.3.2                    pypi_0    pypi
urllib3                   1.24.1                   py36_2    intel
wcwidth                   0.1.7                    pypi_0    pypi
werkzeug                  0.15.2                   pypi_0    pypi
wheel                     0.31.0                   py36_3    intel
widgetsnbextension        3.4.2                    pypi_0    pypi
xz                        5.2.3                         2    intel
yaml                      0.1.7                         2    intel
zlib                      1.2.11                        5    intel

Conda info:

active environment : idp
    active env location : /anaconda3/envs/idp
            shell level : 3
       user config file : /Users/adam/.condarc
 populated config files : /Users/adam/.condarc
          conda version : 4.6.4
    conda-build version : 3.17.6
         python version : 3.7.1.final.0
       base environment : /anaconda3  (writable)
           channel URLs : https://conda.anaconda.org/intel/osx-64
                          https://conda.anaconda.org/intel/noarch
                          https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /anaconda3/pkgs
                          /Users/adam/.conda/pkgs
       envs directories : /anaconda3/envs
                          /Users/adam/.conda/envs
               platform : osx-64
             user-agent : conda/4.6.4 requests/2.21.0 CPython/3.7.1 Darwin/18.5.0 OSX/10.14.4
                UID:GID : 501:20
             netrc file : None
           offline mode : False

 

Todd_T_Intel
Employee
455 Views

Hello,

From the output of "conda list" it appears that many of the Intel packages were replaced by packages from PyPI, including numpy. The latest version of numpy in the Intel channel on anaconda.org is 1.16.1, and your version is 1.14.5. The pandas package in your environment was built with numpy 1.16 using Intel(R) MKL, while the numpy 1.14.5 in your environment came from PyPI.

My guess is that after creation of your initial environment, you installed Tensorflow from PyPI, and many of the conda-installed packages were overwritten.

My recommendation is to use conda to install tensorflow from the Anaconda default channel using "conda install tensorflow". You will get a much more recent version of Tensorflow (1.13.1 instead of 1.7.1) accelerated with Intel(R) MKL-DNN and your other Intel packages will be preserved.

-Todd

A__Adham
Beginner
455 Views

Thank you. Only that Unity ML Agents does not support above numpy 1.14

Todd_T_Intel
Employee
455 Views

Adham,

If you require specific older versions of packages in order to be compatible with other software such as Unity ML Agents, you will need to be more explicit with versions when you create your environment. It is also more tricky when you are mixing package managers (pip and conda).

My recommendation, then, is to create an environment that specifies all the version requirements that you are aware of. For example:

$ conda create -n unityml -c intel python=3.6 numpy=1.14 scipy pandas
Collecting package metadata: done
Solving environment: done

....

The following NEW packages will be INSTALLED:

  bzip2              intel/osx-64::bzip2-1.0.6-17
  certifi            intel/osx-64::certifi-2018.1.18-py36_2
  cycler             intel/osx-64::cycler-0.10.0-py36_7
  freetype           intel/osx-64::freetype-2.9-3
  icc_rt             intel/osx-64::icc_rt-2019.3-intel_199
  intel-openmp       intel/osx-64::intel-openmp-2019.3-intel_199
  intelpython        intel/osx-64::intelpython-2019.3-0
  kiwisolver         intel/osx-64::kiwisolver-1.0.1-py36_2
  libpng             intel/osx-64::libpng-1.6.36-2
  matplotlib         intel/osx-64::matplotlib-2.2.2-np114py36_4
  mkl                intel/osx-64::mkl-2019.3-intel_199
  mkl_fft            intel/osx-64::mkl_fft-1.0.2-np114py36_6
  mkl_random         intel/osx-64::mkl_random-1.0.1-np114py36_6
  numexpr            intel/osx-64::numexpr-2.6.5-np114py36_2
  numpy              intel/osx-64::numpy-1.14.3-py36_5
  openssl            intel/osx-64::openssl-1.0.2r-0
  pandas             intel/osx-64::pandas-0.23.0-np114py36_1
  pip                intel/osx-64::pip-10.0.1-py36_0
  pyparsing          intel/osx-64::pyparsing-2.2.0-py36_2
  python             intel/osx-64::python-3.6.8-6
  python-dateutil    intel/osx-64::python-dateutil-2.6.0-py36_12
  pytz               intel/osx-64::pytz-2018.4-py36_3
  scipy              intel/osx-64::scipy-1.1.0-np114py36_6
  setuptools         intel/osx-64::setuptools-39.0.1-py36_0
  six                intel/osx-64::six-1.11.0-py36_3
  sqlite             intel/osx-64::sqlite-3.27.2-2
  tbb                intel/osx-64::tbb-2019.4-intel_199
  tbb4py             intel/osx-64::tbb4py-2019.4-py36_intel_0
  tcl                intel/osx-64::tcl-8.6.4-22
  tk                 intel/osx-64::tk-8.6.4-28
  wheel              intel/osx-64::wheel-0.31.0-py36_3
  xz                 intel/osx-64::xz-5.2.3-2
  zlib               intel/osx-64::zlib-1.2.11-5

Proceed (/n)?

-----------------------------

Then, when installing additional packages from PyPI, ensure that none of your existing packages are overwritten.

Again, mixing packages from pip and conda can be tricky, so it may take some time and a few iterations to get it right.

 

Hope this helps,

Todd

 

Reply