AI Tools from Intel
Find answers to your toolkit installation, configuration, and get-started questions.
125 Discussions

Error occurred when use ' conda create -y -v --clone ' to clone pytorch, but tensorflow not.

mpi_new_user
New Contributor II
‎05-10-2021 07:50 PM
3,416 Views

Hi，

I run the image which is built by singularity and based on Intel AI Toolkit.

When I run the command  `conda env list`:

root@head:/home/hpcadmin# conda env list
# conda environments:
#
                         /home/hpcadmin/fy/test_ai/tensorflow2
base                  *  /opt/intel/oneapi/intelpython/latest
2021.2.0                 /opt/intel/oneapi/intelpython/latest/envs/2021.2.0
pytorch                  /opt/intel/oneapi/intelpython/latest/envs/pytorch
pytorch-1.7.0            /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.7.0
tensorflow               /opt/intel/oneapi/intelpython/latest/envs/tensorflow
tensorflow-2.3.0         /opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.3.0
                         /tmp/hpcadmin/oneapi/pytorch/latest

 And  the error will happen after I run the clone command:

 

root@head:/home/hpcadmin/fy/test_ai# conda create -y -v --clone pytorch -p /home/hpcadmin/fy/test_ai/pytorch
Source:      /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.7.0
Destination: /home/hpcadmin/fy/test_ai/pytorch
The following packages cannot be cloned out of the root environment:
 - defaults/linux-64::conda-4.10.1-py37h06a4308_1
Packages: 145
Files: 203

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/cli/main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 83, in do_call
        return getattr(module, func_name)(args, parser)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/cli/main_create.py", line 41, in execute
        install(args, parser, 'create')
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/cli/install.py", line 222, in install
        clone(args.clone, prefix, json=context.json, quiet=context.quiet, index_args=index_args)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/cli/install.py", line 74, in clone
        index_args=index_args)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/misc.py", line 290, in clone_env
        force_extract=False, index_args=index_args)
      File "/opt/intel/oneapi/intelpython/latest/lib/python3.7/site-packages/conda/misc.py", line 90, in explicit
        assert not any(spec_pcrec[1] is None for spec_pcrec in specs_pcrecs)
    AssertionError

`$ /opt/intel/oneapi/intelpython/latest/bin/conda create -y -v --clone pytorch-1.7.0 -p /home/hpcadmin/fy/test_ai/pytorch`

  environment variables:
    ACL_BOARD_VENDOR_PATH=/opt/Intel/OpenCLFPGA/oneAPI/Boards
                 CIO_TEST=<not set>
                CLASSPATH=/opt/intel/oneapi/mpi/2021.2.0//lib/mpi.jar:/opt/intel/oneapi/dal/2021
                          .2.0/lib/onedal.jar
        CMAKE_PREFIX_PATH=/opt/intel/oneapi/vpl:/opt/intel/oneapi/tbb/2021.2.0/env/..:/opt/intel
                          /oneapi/dal/2021.2.0
                CONDA_EXE=/opt/intel/oneapi/intelpython/latest/bin/conda
         CONDA_PYTHON_EXE=/opt/intel/oneapi/intelpython/latest/bin/python
               CONDA_ROOT=/opt/intel/oneapi/intelpython/latest
              CONDA_SHLVL=0
                    CPATH=/opt/intel/oneapi/vpl/2021.2.2/include:/opt/intel/oneapi/tbb/2021.2.0/
                          env/../include:/opt/intel/oneapi/mpi/2021.2.0//include:/opt/intel/onea
                          pi/mkl/latest/include:/opt/intel/oneapi/ippcp/2021.2.0/include:/opt/in
                          tel/oneapi/ipp/2021.2.0/include:/opt/intel/oneapi/dpl/2021.2.0/linux/i
                          nclude:/opt/intel/oneapi/dnnl/2021.2.0/cpu_dpcpp_gpu_dpcpp/lib:/opt/in
                          tel/oneapi/dev-utilities/2021.2.0/include:/opt/intel/oneapi/dal/2021.2
                          .0/include:/opt/intel/oneapi/compiler/2021.2.0/linux/include:/opt/inte
                          l/oneapi/ccl/2021.2.0/include/cpu_gpu_dpcpp
           CURL_CA_BUNDLE=<not set>
         FI_PROVIDER_PATH=/opt/intel/oneapi/mpi/2021.2.0/libfabric/lib/prov:/usr/lib64/libfabric
                 INFOPATH=/opt/intel/oneapi/debugger/10.1.1/documentation/info/
          LD_LIBRARY_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib:/opt/ohpc/pub/mpi/ucx-ohpc/1.8.
                          0/lib:/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib:/opt/ohpc/pub/compiler
                          /gcc/9.3.0/lib64:/.singularity.d/libs
             LIBRARY_PATH=/opt/intel/oneapi/vpl/2021.2.2/lib:/opt/intel/oneapi/tbb/2021.2.0/env/
                          ../lib/intel64/gcc4.8:/opt/intel/oneapi/mpi/2021.2.0//libfabric/lib:/o
                          pt/intel/oneapi/mpi/2021.2.0//lib/release:/opt/intel/oneapi/mpi/2021.2
                          .0//lib:/opt/intel/oneapi/mkl/latest/lib/intel64:/opt/intel/oneapi/ipp
                          cp/2021.2.0/lib/intel64:/opt/intel/oneapi/ipp/2021.2.0/lib/intel64:/op
                          t/intel/oneapi/dnnl/2021.2.0/cpu_dpcpp_gpu_dpcpp/lib:/opt/intel/oneapi
                          /dal/2021.2.0/lib/intel64:/opt/intel/oneapi/compiler/2021.2.0/linux/co
                          mpiler/lib/intel64_lin:/opt/intel/oneapi/compiler/2021.2.0/linux/lib:/
                          opt/intel/oneapi/ccl/2021.2.0/lib/cpu_gpu_dpcpp
                  MANPATH=/opt/xcat/share/man:/opt/ohpc/pub/libs/singularity/3.4.2/share/man:/op
                          t/ohpc/pub/mpi/libfabric/1.10.1/share/man:/opt/ohpc/pub/mpi/openmpi4-g
                          nu9/4.0.4/share/man:/opt/ohpc/pub/compiler/gcc/9.3.0/share/man:/usr/lo
                          cal/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/ma
                          n:/opt/confluent/share/man::
               MODULEPATH=/opt/ohpc/pub/moduledeps/gnu9-openmpi4:/opt/ohpc/pub/moduledeps/gnu9:/
                          opt/ohpc/admin/modulefiles:/opt/ohpc/pub/modulefiles:/opt/intel/oneapi
                          /modulefiles
                  NLSPATH=/opt/intel/oneapi/mkl/latest/lib/intel64/locale/%l_%t/%N
                     PATH=/opt/intel/oneapi/intelpython/latest/bin:/opt/intel/oneapi/intelpython
                          /latest/bin:/opt/intel/oneapi/vtune/2021.2.0/bin64:/opt/intel/oneapi/v
                          pl/2021.2.2/bin:/opt/intel/oneapi/mpi/2021.2.0/libfabric/bin:/opt/inte
                          l/oneapi/mpi/2021.2.0/bin:/opt/intel/oneapi/mkl/latest/bin/intel64:/op
                          t/intel/oneapi/intelpython/latest/bin:/opt/intel/oneapi/intelpython/la
                          test/condabin:/opt/intel/oneapi/dev-utilities/2021.2.0/bin:/opt/intel/
                          oneapi/debugger/10.1.1/gdb/intel64/bin:/opt/intel/oneapi/compiler/2021
                          .2.0/linux/lib/oclfpga/llvm/aocl-bin:/opt/intel/oneapi/compiler/2021.2
                          .0/linux/lib/oclfpga/bin:/opt/intel/oneapi/compiler/2021.2.0/linux/bin
                          /intel64:/opt/intel/oneapi/compiler/2021.2.0/linux/bin:/opt/intel/onea
                          pi/compiler/2021.2.0/linux/ioc/bin:/opt/intel/oneapi/advisor/2021.2.0/
                          bin64:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
          PKG_CONFIG_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib/pkgconfig:/opt/ohpc/pub/mpi/ucx
                          -ohpc/1.8.0/lib/pkgconfig:/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib/pk
                          gconfig
               PYTHONPATH=/opt/intel/oneapi/advisor/2021.2.0/pythonapi
       REQUESTS_CA_BUNDLE=<not set>
        SETVARS_VARS_PATH=/opt/intel/oneapi/vtune/latest/env/vars.sh
            SSL_CERT_FILE=<not set>
                USER_PATH=/opt/xcat/bin:/opt/xcat/sbin:/opt/xcat/share/xcat/tools:/opt/ohpc/pub/
                          libs/singularity/3.4.2/bin:/opt/ohpc/pub/mpi/libfabric/1.10.1/bin:/opt
                          /ohpc/pub/mpi/ucx-ohpc/1.8.0/bin:/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4
                          /bin:/opt/ohpc/pub/compiler/gcc/9.3.0/bin:/opt/ohpc/pub/utils/prun/2.0
                          :/opt/ohpc/pub/bin:/opt/confluent/bin:/usr/local/sbin:/usr/local/bin:/
                          usr/sbin:/usr/bin:/root/bin:/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/b
                          in:/usr/local/sbin
__LMOD_REF_COUNT_LD_LIBRARY_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib:1;/opt/ohpc/pub/mpi/ucx-ohpc/1.
                          8.0/lib:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib:1;/opt/ohpc/pub/co
                          mpiler/gcc/9.3.0/lib64:1
 __LMOD_REF_COUNT_MANPATH=/opt/ohpc/pub/libs/singularity/3.4.2/share/man:1;/opt/ohpc/pub/mpi/lib
                          fabric/1.10.1/share/man:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/share/
                          man:1;/opt/ohpc/pub/compiler/gcc/9.3.0/share/man:1;/usr/local/share/ma
                          n:1;/usr/share/man/overrides:1;/usr/share/man/en:1;/usr/share/man:1;/o
                          pt/confluent/share/man:1
__LMOD_REF_COUNT_MODULEPATH=/opt/ohpc/pub/moduledeps/gnu9-openmpi4:1;/opt/ohpc/pub/moduledeps/gnu9
                          :1;/opt/ohpc/admin/modulefiles:1;/opt/ohpc/pub/modulefiles:1;/opt/inte
                          l/oneapi/modulefiles:1
    __LMOD_REF_COUNT_PATH=/opt/ohpc/pub/libs/singularity/3.4.2/bin:1;/opt/ohpc/pub/mpi/libfabric
                          /1.10.1/bin:1;/opt/ohpc/pub/mpi/ucx-ohpc/1.8.0/bin:1;/opt/ohpc/pub/mpi
                          /openmpi4-gnu9/4.0.4/bin:1;/opt/ohpc/pub/compiler/gcc/9.3.0/bin:1;/opt
                          /ohpc/pub/utils/prun/2.0:1;/opt/ohpc/pub/bin:1;/opt/confluent/bin:1;/u
                          sr/local/sbin:1;/usr/local/bin:1;/usr/sbin:1;/usr/bin:1
__LMOD_REF_COUNT_PKG_CONFIG_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib/pkgconfig:1;/opt/ohpc/pub/mpi/u
                          cx-ohpc/1.8.0/lib/pkgconfig:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/li
                          b/pkgconfig:1

     active environment : None
            shell level : 0
       user config file : /root/.condarc
 populated config files : /opt/intel/oneapi/intelpython/latest/.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.7.9.final.0
       virtual packages : __glibc=2.27=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /opt/intel/oneapi/intelpython/latest  (read only)
           channel URLs : file:///opt/intel/oneapi/conda_channel/linux-64
                          file:///opt/intel/oneapi/conda_channel/noarch
                          https://conda.anaconda.org/intel/linux-64
                          https://conda.anaconda.org/intel/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/intel/oneapi/intelpython/latest/pkgs
                          /root/.conda/pkgs
       envs directories : /root/.conda/envs
                          /opt/intel/oneapi/intelpython/latest/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.25.1 CPython/3.7.9 Linux/4.18.0-193.el8.x86_64 ubuntu/18.04.5 glibc/2.27
                UID:GID : 0:0
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

Upload successful.

 

 

But,it is successful when I clone the tensorflow environment.

 

initializing UnlinkLinkTransaction with
  target_prefix: /home/hpcadmin/fy/test_ai/tensorflow
  unlink_precs:

  link_precs:
    defaults/linux-64::ca-certificates-2021.4.13-h06a4308_1
    file:///opt/intel/oneapi/conda_channel/linux-64::common_cmplr_lib_rt-2021.2.0-intel_610
    file:///opt/intel/oneapi/conda_channel/linux-64::common_cmplr_lic_rt-2021.2.0-intel_610
    file:///opt/intel/oneapi/conda_channel/linux-64::impi_rt-2021.2.0-intel_215
    file:///opt/intel/oneapi/conda_channel/linux-64::intel-openmp-2021.2.0-intel_610
    file:///opt/intel/oneapi/conda_channel/linux-64::intelpython-2021.1.1-1
    file:///opt/intel/oneapi/conda_channel/linux-64::libgcc-ng-9.3.0-hdf63c60_101
    file:///opt/intel/oneapi/conda_channel/linux-64::libstdcxx-ng-9.3.0-hdf63c60_101
    defaults/linux-64::pandoc-2.12-h06a4308_0
    file:///opt/intel/oneapi/conda_channel/linux-64::tbb-2021.2.0-intel_357
    file:///opt/intel/oneapi/conda_channel/linux-64::tcl-8.6.10-0
    file:///opt/intel/oneapi/conda_channel/linux-64::tensorflow-base-2.3.0-0
    file:///opt/intel/oneapi/conda_channel/linux-64::bzip2-1.0.8-hb9a14ef_6
    file:///opt/intel/oneapi/conda_channel/linux-64::c-ares-1.17.1-h14c3975_0
    defaults/linux-64::expat-2.3.0-h2531618_2
    file:///opt/intel/oneapi/conda_channel/linux-64::fortran_rt-2021.2.0-intel_610
    defaults/linux-64::icu-58.2-he6710b0_3
    defaults/linux-64::jpeg-9b-h024ee3a_2
    file:///opt/intel/oneapi/conda_channel/linux-64::libffi-3.3-13
    defaults/linux-64::libsodium-1.0.18-h7b6447c_0
    defaults/linux-64::libuuid-1.0.3-h1bed415_2
    defaults/linux-64::libxcb-1.14-h7b6447c_0
    file:///opt/intel/oneapi/conda_channel/linux-64::lz4-c-1.9.2-h5c78bee_3
    file:///opt/intel/oneapi/conda_channel/linux-64::lzo-2.10-hb9a14ef_6
    file:///opt/intel/oneapi/conda_channel/linux-64::mkl-2021.2.0-intel_296
    file:///opt/intel/oneapi/conda_channel/linux-64::opencl_rt-2021.2.0-intel_610
    defaults/linux-64::openssl-1.1.1k-h27cfd23_0
    defaults/linux-64::pcre-8.44-he6710b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::sqlite-3.34.0-h2132df7_0
    file:///opt/intel/oneapi/conda_channel/linux-64::xz-5.2.5-h85f8e93_2
    file:///opt/intel/oneapi/conda_channel/linux-64::yaml-0.1.7-7
    file:///opt/intel/oneapi/conda_channel/linux-64::zlib-1.2.11.1-h047b5d8_3
    defaults/linux-64::glib-2.68.1-h36276a3_0
    file:///opt/intel/oneapi/conda_channel/linux-64::hdf5-1.10.2-3
    defaults/linux-64::libpng-1.6.37-hbc83047_0
    file:///opt/intel/oneapi/conda_channel/linux-64::libprotobuf-3.14.0-hdfd0036_0
    file:///opt/intel/oneapi/conda_channel/linux-64::libxml2-2.9.10-h58efa85_4
    file:///opt/intel/oneapi/conda_channel/linux-64::tk-8.6.10-h8e2d9d6_2
    defaults/linux-64::zeromq-4.3.4-h2531618_0
    file:///opt/intel/oneapi/conda_channel/linux-64::zstd-1.4.5-ha6fa1a4_0
    defaults/linux-64::dbus-1.13.18-hb2f20db_0
    defaults/linux-64::freetype-2.10.4-h5ab3b9f_0
    defaults/linux-64::gstreamer-1.14.0-h28cd5cc_2
    file:///opt/intel/oneapi/conda_channel/linux-64::libarchive-3.4.2-hae7a978_7
    file:///opt/intel/oneapi/conda_channel/linux-64::python-3.7.9-h86c8c07_7
    file:///opt/intel/oneapi/conda_channel/linux-64::asn1crypto-1.4.0-py37_2
    file:///opt/intel/oneapi/conda_channel/linux-64::async-timeout-3.0.1-py37h06a4308_0
    defaults/linux-64::async_generator-1.10-py37h28b3542_0
    file:///opt/intel/oneapi/conda_channel/linux-64::attrs-20.3.0-pyhd3eb1b0_0
    defaults/noarch::backcall-0.2.0-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::blinker-1.4-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::cachetools-4.2.1-pyhd3eb1b0_0
    defaults/linux-64::certifi-2020.12.5-py37h06a4308_0
    file:///opt/intel/oneapi/conda_channel/linux-64::chardet-4.0.0-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::click-7.1.2-py_0
    defaults/noarch::decorator-5.0.6-pyhd3eb1b0_0
    defaults/noarch::defusedxml-0.7.1-pyhd3eb1b0_0
    defaults/linux-64::entrypoints-0.3-py37_0
    defaults/linux-64::fontconfig-2.13.1-h6c09931_0
    file:///opt/intel/oneapi/conda_channel/linux-64::gast-0.3.3-py_0
    defaults/linux-64::gst-plugins-base-1.14.0-h8213a91_2
    file:///opt/intel/oneapi/conda_channel/linux-64::idna-3.1-py37_0
    defaults/noarch::ipython_genutils-0.2.0-pyhd3eb1b0_1
    defaults/noarch::jupyterlab_widgets-1.0.0-pyhd3eb1b0_1
    defaults/linux-64::markupsafe-1.1.1-py37h14c3975_1
    defaults/linux-64::mistune-0.8.4-py37h14c3975_1001
    file:///opt/intel/oneapi/conda_channel/linux-64::multidict-4.7.6-py37h7b6447c_1
    defaults/noarch::nest-asyncio-1.5.1-pyhd3eb1b0_0
    defaults/linux-64::pandocfilters-1.4.3-py37h06a4308_1
    defaults/noarch::parso-0.8.2-pyhd3eb1b0_0
    defaults/noarch::pickleshare-0.7.5-pyhd3eb1b0_1003
    defaults/noarch::prometheus_client-0.10.1-pyhd3eb1b0_0
    defaults/noarch::ptyprocess-0.7.0-pyhd3eb1b0_2
    file:///opt/intel/oneapi/conda_channel/linux-64::pyasn1-0.4.8-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pycosat-0.6.3-py37_6
    file:///opt/intel/oneapi/conda_channel/linux-64::pycparser-2.20-py37_2
    defaults/noarch::pyparsing-2.4.7-pyhd3eb1b0_0
    defaults/linux-64::pyrsistent-0.17.3-py37h7b6447c_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pysocks-1.7.0-py37_2
    file:///opt/intel/oneapi/conda_channel/linux-64::python-libarchive-c-2.8-py37_14
    defaults/linux-64::pyzmq-20.0.0-py37h2531618_1
    defaults/noarch::qtpy-1.9.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::ruamel_yaml-0.15.99-py37h725dc98_6
    defaults/noarch::send2trash-1.5.0-pyhd3eb1b0_1
    defaults/linux-64::sip-4.19.8-py37hf484d3e_0
    file:///opt/intel/oneapi/conda_channel/linux-64::six-1.15.0-py37_1
    file:///opt/intel/oneapi/conda_channel/linux-64::tbb4py-2021.2.0-py37_intel_357
    file:///opt/intel/oneapi/conda_channel/linux-64::tensorflow-estimator-2.3.0-pyheb71bc4_0
    file:///opt/intel/oneapi/conda_channel/linux-64::termcolor-1.1.0-py37_1
    defaults/noarch::testpath-0.4.4-pyhd3eb1b0_0
    defaults/linux-64::tornado-6.1-py37h27cfd23_0
    file:///opt/intel/oneapi/conda_channel/linux-64::tqdm-4.56.2-py37_0
    defaults/noarch::wcwidth-0.2.5-py_0
    defaults/linux-64::webencodings-0.5.1-py37_1
    file:///opt/intel/oneapi/conda_channel/linux-64::werkzeug-1.0.1-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::wrapt-1.12.1-py37h7b6447c_1
    file:///opt/intel/oneapi/conda_channel/linux-64::zipp-3.4.0-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::absl-py-0.11.0-pyhd3eb1b0_1
    file:///opt/intel/oneapi/conda_channel/linux-64::astunparse-1.6.3-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::cffi-1.14.4-py37h14c3975_0
    file:///opt/intel/oneapi/conda_channel/linux-64::conda-package-handling-1.4.1-py37_4
    file:///opt/intel/oneapi/conda_channel/linux-64::google-pasta-0.2.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::importlib-metadata-2.0.0-py_1
    defaults/linux-64::jedi-0.17.0-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::mkl-service-2.3.0-py37hb9838c7_10
    defaults/noarch::packaging-20.9-pyhd3eb1b0_0
    defaults/noarch::pexpect-4.8.0-pyhd3eb1b0_3
    defaults/noarch::prompt-toolkit-3.0.17-pyh06a4308_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pyasn1-modules-0.2.8-py_0
    defaults/noarch::python-dateutil-2.8.1-pyhd3eb1b0_0
    defaults/linux-64::qt-5.9.7-h5867ecd_1
    file:///opt/intel/oneapi/conda_channel/linux-64::rsa-4.7-pyhd3eb1b0_1
    file:///opt/intel/oneapi/conda_channel/linux-64::setuptools-51.1.2-py37_0
    defaults/linux-64::terminado-0.9.4-py37h06a4308_0
    defaults/noarch::traitlets-5.0.5-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::yarl-1.6.3-py37h27cfd23_0
    file:///opt/intel/oneapi/conda_channel/linux-64::aiohttp-3.6.3-py37h7b6447c_0
    defaults/linux-64::argon2-cffi-20.1.0-py37h27cfd23_1
    defaults/noarch::bleach-3.3.0-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::cryptography-3.3.2-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::grpcio-1.31.0-py37hf8bcb03_0
    defaults/noarch::jinja2-2.11.3-pyhd3eb1b0_0
    defaults/linux-64::jsonschema-3.0.2-py37_0
    defaults/linux-64::jupyter_core-4.7.1-py37h06a4308_0
    file:///opt/intel/oneapi/conda_channel/linux-64::markdown-3.3.3-py37h06a4308_0
    defaults/noarch::prompt_toolkit-3.0.17-hd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::protobuf-3.14.0-py37h2531618_1
    defaults/noarch::pygments-2.8.1-pyhd3eb1b0_0
    defaults/linux-64::pyqt-5.9.2-py37h05f1152_2
    file:///opt/intel/oneapi/conda_channel/linux-64::tensorboard-plugin-wit-1.6.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::wheel-0.36.2-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::google-auth-1.24.0-pyhd3eb1b0_0
    defaults/linux-64::ipython-7.22.0-py37hb070fc8_0
    defaults/noarch::jupyter_client-6.1.12-pyhd3eb1b0_0
    defaults/noarch::jupyterlab_pygments-0.1.2-py_0
    defaults/noarch::nbformat-5.1.3-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pip-21.0.1-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pyjwt-1.7.1-py37_0
    file:///opt/intel/oneapi/conda_channel/linux-64::pyopenssl-20.0.1-py37_2
    defaults/linux-64::ipykernel-5.3.4-py37h5ca1d4c_0
    defaults/noarch::nbclient-0.5.3-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::oauthlib-3.1.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::urllib3-1.26.3-py37_0
    defaults/noarch::jupyter_console-6.4.0-pyhd3eb1b0_0
    defaults/linux-64::nbconvert-6.0.7-py37_0
    defaults/noarch::qtconsole-5.0.3-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::requests-2.25.1-py37_1
    defaults/linux-64::notebook-6.3.0-py37h06a4308_0
    file:///opt/intel/oneapi/conda_channel/linux-64::requests-oauthlib-1.3.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::google-auth-oauthlib-0.4.2-pyhd3eb1b0_2
    defaults/linux-64::widgetsnbextension-3.5.1-py37_0
    defaults/noarch::ipywidgets-7.6.3-pyhd3eb1b0_1
    defaults/linux-64::jupyter-1.0.0-py37_7
    file:///opt/intel/oneapi/conda_channel/linux-64::dpcpp_cpp_rt-2021.2.0-intel_610
    file:///opt/intel/oneapi/conda_channel/linux-64::icc_rt-2021.2.0-intel_610
    file:///opt/intel/oneapi/conda_channel/linux-64::numpy-base-1.20.1-py37hffe281f_0
    file:///opt/intel/oneapi/conda_channel/linux-64::mkl_fft-1.3.0-py37h9bd69cd_0
    file:///opt/intel/oneapi/conda_channel/linux-64::mkl_random-1.2.1-py37h3617918_2
    file:///opt/intel/oneapi/conda_channel/linux-64::mkl_umath-0.1.1-py37hc4b4d23_8
    file:///opt/intel/oneapi/conda_channel/linux-64::numpy-1.20.1-py37h288f0c4_0
    file:///opt/intel/oneapi/conda_channel/linux-64::h5py-2.8.0-py37h989c5e5_3
    file:///opt/intel/oneapi/conda_channel/linux-64::opt_einsum-3.1.0-py_0
    file:///opt/intel/oneapi/conda_channel/linux-64::scipy-1.6.1-py37haaffb16_2
    file:///opt/intel/oneapi/conda_channel/linux-64::tensorboard-2.3.0-pyh4dce500_0
    file:///opt/intel/oneapi/conda_channel/linux-64::keras-preprocessing-1.1.2-pyhd3eb1b0_0
    file:///opt/intel/oneapi/conda_channel/linux-64::tensorflow-2.3.0-py37_0


Source:      /opt/intel/oneapi/intelpython/latest/envs/tensorflow
Destination: /home/hpcadmin/fy/test_ai/tensorflow
The following packages cannot be cloned out of the root environment:
 - defaults/linux-64::conda-4.10.1-py37h06a4308_1
Packages: 166
Files: 204
Preparing transaction: ...working... done
Verifying transaction: ...working... [ClobberError: This transaction has incompatible packages due to a shared path.
  packages: file:///opt/intel/oneapi/conda_channel/linux-64::tensorboard-2.3.0-pyh4dce500_0, file:///opt/intel/oneapi/conda_channel/linux-64::tensorflow-2.3.0-py37_0
  path: 'bin/tensorboard'

]

done
Executing transaction: ...working... ===> LINKING PACKAGE: defaults::ca-certificates-2021.4.13-h06a4308_1 <===
  prefix=/home/hpcadmin/fy/test_ai/tensorflow
  source=/root/.conda/pkgs/ca-certificates-2021.4.13-h06a4308_1


===> LINKING PACKAGE: file:///opt/intel/oneapi/conda_channel::common_cmplr_lib_rt-2021.2.0-intel_610 <===
  prefix=/home/hpcadmin/fy/test_ai/tensorflow
  source=/opt/intel/oneapi/intelpython/latest/pkgs/common_cmplr_lib_rt-2021.2.0-intel_610

 

 

Please let me know if I'm missing something.Thanks.

 
 
0 Kudos

Link Copied

5 Replies
JananiC_Intel
Moderator
‎05-11-2021 06:35 AM
3,400 Views

Hi,


Thanks for posting in Intel forums.


We will check on this meanwhile could you share your steps followed for building AI toolkit image?


Regards,

Janani Chandran


0 Kudos
Copy link
mpi_new_user
New Contributor II
‎05-11-2021 06:13 PM
3,387 Views
In response to JananiC_Intel

Hi,

I found this error also occurred in the conda environment which is installed directly by Intel AI Toolkit.

Here is the info :

[root@head ~]# cd /opt/intel/oneapi/
[root@head oneapi]# cd pytorch/
[root@head pytorch]# ls
1.5.0  latest
[root@head pytorch]#
[root@head pytorch]# cd latest/
[root@head latest]# ls
bin       conda-meta  etc      install_pytorch.sh  libexec    pkgs   shell  sys_check        x86_64-conda_cos6-linux-gnu
condabin  env         include  lib                 licensing  share  ssl    Untitled6.ipynb
[root@head latest]# cd bin
[root@head bin]#
[root@head bin]#
[root@head bin]# ls
2to3                    fax2ps                    jupyter-run              openssl            sqlite3     wish8.6
2to3-3.7                fax2tiff                  jupyter-serverextension  pal2rgb            tclsh8.6    wrjpgcom
activate                freetype-config           jupyter-troubleshoot     pip                tiff2bw     xml2-config
bsdcat                  idle3                     jupyter-trust            pngfix             tiff2pdf    xmlcatalog
bsdcpio                 idle3.7                   libpng16-config          png-fix-itxt       tiff2ps     xmllint
bsdtar                  iptest                    libpng-config            ppm2tiff           tiff2rgba   xz
chardetect              iptest3                   lz4                      pydoc              tiffcmp     xzcat
cjpeg                   ipython                   lz4c                     pydoc3             tiffcp      xzcmp
conda                   ipython3                  lz4cat                   pydoc3.7           tiffcrop    xzdec
conda-env               jpegtran                  lzcat                    pygmentize         tiffdither  xzdiff
convert-caffe2-to-onnx  jsonschema                lzcmp                    python             tiffdump    xzegrep
convert-onnx-to-caffe2  jupyter                   lzdiff                   python3            tiffinfo    xzfgrep
cph                     jupyter-bundlerextension  lzegrep                  python3.7          tiffmedian  xzgrep
c_rehash                jupyter-console           lzfgrep                  python3.7-config   tiffset     xzless
deactivate              jupyter-kernel            lzgrep                   python3.7m         tiffsplit   xzmore
djpeg                   jupyter-kernelspec        lzless                   python3.7m-config  tqdm        zstd
easy_install            jupyter-migrate           lzma                     python3-config     unlz4       zstdcat
easy_install-3.7        jupyter-nbconvert         lzmadec                  pyvenv             unlzma      zstdgrep
f2py                    jupyter-nbextension       lzmainfo                 pyvenv-3.7         unxz        zstdless
f2py3                   jupyter-notebook          lzmore                   raw2tiff           unzstd      zstdmt
f2py3.7                 jupyter-qtconsole         ninja                    rdjpgcom           wheel
[root@head bin]#
[root@head bin]#
[root@head bin]#
[root@head bin]# source activate
(base) [root@head bin]#
(base) [root@head bin]#
(base) [root@head bin]# conda env list
# conda environments:
#
                         /home/hpcadmin/conda
                         /home/hpcadmin/devtool_conda
                         /home/hpcadmin/fy/test_ai/pytorch
                         /home/hpcadmin/fy/test_ai/tensorflow
                         /home/hpcadmin/jupyter_test
                         /home/hpcadmin/test_tensor/pytorch
                         /home/hpcadmin/test_tensor/tensorflow
base                  *  /opt/intel/oneapi/pytorch/latest
                         /opt/miniconda
                         /tmp/hpcadmin/oneapi/pytorch/latest

(base) [root@head bin]# conda create -y -v --clone base -p /opt/pytorch
Source:      /opt/intel/oneapi/pytorch/latest
Destination: /opt/pytorch
The following packages cannot be cloned out of the root environment:
 - file:///opt/intel/oneapi/conda_channel/linux-64::conda-4.8.4-py37h926af72_10
Packages: 68
Files: 10673

# >>>>>>>>>>>>>>>>>>>>>> ERROR REPORT <<<<<<<<<<<<<<<<<<<<<<

    Traceback (most recent call last):
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/exceptions.py", line 1079, in __call__
        return func(*args, **kwargs)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/cli/main.py", line 84, in _main
        exit_code = do_call(args, p)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/cli/conda_argparse.py", line 83, in do_call
        return getattr(module, func_name)(args, parser)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/cli/main_create.py", line 41, in execute
        install(args, parser, 'create')
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/cli/install.py", line 222, in install
        clone(args.clone, prefix, json=context.json, quiet=context.quiet, index_args=index_args)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/cli/install.py", line 74, in clone
        index_args=index_args)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/misc.py", line 286, in clone_env
        force_extract=False, index_args=index_args)
      File "/opt/intel/oneapi/pytorch/latest/lib/python3.7/site-packages/conda/misc.py", line 90, in explicit
        assert not any(spec_pcrec[1] is None for spec_pcrec in specs_pcrecs)
    AssertionError

`$ /opt/intel/oneapi/pytorch/latest/bin/conda create -y -v --clone base -p /opt/pytorch`

  environment variables:
                 CIO_TEST=<not set>
        CONDA_DEFAULT_ENV=base
                CONDA_EXE=/opt/intel/oneapi/pytorch/latest/bin/conda
             CONDA_PREFIX=/opt/intel/oneapi/pytorch/latest
    CONDA_PROMPT_MODIFIER=(base)
         CONDA_PYTHON_EXE=/opt/intel/oneapi/pytorch/latest/bin/python
               CONDA_ROOT=/opt/intel/oneapi/pytorch/latest
              CONDA_SHLVL=1
           CURL_CA_BUNDLE=<not set>
          LD_LIBRARY_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib:/opt/ohpc/pub/mpi/ucx-ohpc/1.8.
                          0/lib:/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib:/opt/ohpc/pub/compiler
                          /gcc/9.3.0/lib64
                  MANPATH=/opt/xcat/share/man:/opt/ohpc/pub/libs/singularity/3.4.2/share/man:/op
                          t/ohpc/pub/mpi/libfabric/1.10.1/share/man:/opt/ohpc/pub/mpi/openmpi4-g
                          nu9/4.0.4/share/man:/opt/ohpc/pub/compiler/gcc/9.3.0/share/man:/usr/lo
                          cal/share/man:/usr/share/man/overrides:/usr/share/man/en:/usr/share/ma
                          n:/opt/confluent/share/man::
               MODULEPATH=/opt/ohpc/pub/moduledeps/gnu9-openmpi4:/opt/ohpc/pub/moduledeps/gnu9:/
                          opt/ohpc/admin/modulefiles:/opt/ohpc/pub/modulefiles:/opt/intel/oneapi
                          /modulefiles
                     PATH=/opt/intel/oneapi/pytorch/latest/bin:/opt/intel/oneapi/pytorch/latest/
                          bin:/opt/miniconda/condabin:/opt/xcat/bin:/opt/xcat/sbin:/opt/xcat/sha
                          re/xcat/tools:/opt/ohpc/pub/libs/singularity/3.4.2/bin:/opt/ohpc/pub/m
                          pi/libfabric/1.10.1/bin:/opt/ohpc/pub/mpi/ucx-ohpc/1.8.0/bin:/opt/ohpc
                          /pub/mpi/openmpi4-gnu9/4.0.4/bin:/opt/ohpc/pub/compiler/gcc/9.3.0/bin:
                          /opt/ohpc/pub/utils/prun/2.0:/opt/ohpc/pub/bin:/opt/confluent/bin:/usr
                          /local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
          PKG_CONFIG_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib/pkgconfig:/opt/ohpc/pub/mpi/ucx
                          -ohpc/1.8.0/lib/pkgconfig:/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib/pk
                          gconfig
       REQUESTS_CA_BUNDLE=<not set>
            SSL_CERT_FILE=<not set>
__LMOD_REF_COUNT_LD_LIBRARY_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib:1;/opt/ohpc/pub/mpi/ucx-ohpc/1.
                          8.0/lib:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/lib:1;/opt/ohpc/pub/co
                          mpiler/gcc/9.3.0/lib64:1
 __LMOD_REF_COUNT_MANPATH=/opt/ohpc/pub/libs/singularity/3.4.2/share/man:1;/opt/ohpc/pub/mpi/lib
                          fabric/1.10.1/share/man:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/share/
                          man:1;/opt/ohpc/pub/compiler/gcc/9.3.0/share/man:1;/usr/local/share/ma
                          n:1;/usr/share/man/overrides:1;/usr/share/man/en:1;/usr/share/man:1;/o
                          pt/confluent/share/man:1
__LMOD_REF_COUNT_MODULEPATH=/opt/ohpc/pub/moduledeps/gnu9-openmpi4:1;/opt/ohpc/pub/moduledeps/gnu9
                          :1;/opt/ohpc/admin/modulefiles:1;/opt/ohpc/pub/modulefiles:1;/opt/inte
                          l/oneapi/modulefiles:1
    __LMOD_REF_COUNT_PATH=/opt/ohpc/pub/libs/singularity/3.4.2/bin:1;/opt/ohpc/pub/mpi/libfabric
                          /1.10.1/bin:1;/opt/ohpc/pub/mpi/ucx-ohpc/1.8.0/bin:1;/opt/ohpc/pub/mpi
                          /openmpi4-gnu9/4.0.4/bin:1;/opt/ohpc/pub/compiler/gcc/9.3.0/bin:1;/opt
                          /ohpc/pub/utils/prun/2.0:1;/opt/ohpc/pub/bin:1;/opt/confluent/bin:1;/u
                          sr/local/sbin:1;/usr/local/bin:1;/usr/sbin:1;/usr/bin:1
__LMOD_REF_COUNT_PKG_CONFIG_PATH=/opt/ohpc/pub/mpi/libfabric/1.10.1/lib/pkgconfig:1;/opt/ohpc/pub/mpi/u
                          cx-ohpc/1.8.0/lib/pkgconfig:1;/opt/ohpc/pub/mpi/openmpi4-gnu9/4.0.4/li
                          b/pkgconfig:1

     active environment : base
    active env location : /opt/intel/oneapi/pytorch/latest
            shell level : 1
       user config file : /root/.condarc
 populated config files : /root/.condarc
          conda version : 4.8.4
    conda-build version : not installed
         python version : 3.7.9.final.0
       virtual packages : __glibc=2.28
       base environment : /opt/intel/oneapi/pytorch/latest  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /opt/intel/oneapi/pytorch/latest/pkgs
                          /root/.conda/pkgs
       envs directories : /opt/intel/oneapi/pytorch/latest/envs
                          /root/.conda/envs
               platform : linux-64
             user-agent : conda/4.8.4 requests/2.24.0 CPython/3.7.9 Linux/4.18.0-193.el8.x86_64 centos/8.2.2004 glibc/2.28
                UID:GID : 0:0
             netrc file : None
           offline mode : False


An unexpected error has occurred. Conda has prepared the above report.

Upload successful.

 

In response to JananiC_Intel
0 Kudos
Copy link
mpi_new_user
New Contributor II
‎05-11-2021 06:18 PM
3,385 Views

And  I found an alternative :

conda env export -n pytorch > pytorch_environment.yml
conda env create -f  pytorch_environment.yml -p  /clone_path

 

I hope it can help to solve this problem。

Thanks.

 

0 Kudos
Copy link
JananiC_Intel
Moderator
‎05-14-2021 06:36 AM
3,346 Views

Hi,

 

Thanks for posting in Intel forums.

 

This is a known issue and we are working on it.

 

There are other methods to use pytorch.You can use pytorch by creating AI toolkit pytorch environment with conda(https://software.intel.com/content/www/us/en/develop/articles/installing-ai-kit-with-conda.html#gs.0sd3qn) and by installing AI analytics toolkit(https://software.intel.com/content/www/us/en/develop/tools/oneapi/ai-analytics-toolkit/download.html?operatingsystem=linux).

 

Regards,

Janani Chandran

 

0 Kudos
Copy link
JananiC_Intel
Moderator
‎12-14-2021 04:20 AM
2,835 Views

Hi,


Sorry for the late response. This issue is been fixed. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.


Regards,

Janani Chandran


0 Kudos
Copy link
Reply

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.