I want to reset intel devcloud

[ffa]

Seems like I broke something

u177925@login-2:~/project$ conda activate pytorch
(pytorch) u177925@login-2:~/project$ python
Python 3.9.13 (main, Oct 19 2022, 17:23:07) 
[GCC 11.2.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import pytorch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'pytorch'
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/u177925/.local/lib/python3.9/site-packages/torch/__init__.py", line 197, in <module>
from torch._C import * # noqa: F403
ImportError: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.26' not found (required by /home/u177925/.local/lib/python3.9/site-packages/torch/lib/libtorch_python.so)
>>> 

 Is there any solution or should I reset?

[RemyaP_Intel]

Hi,

Thank you for posting in Intel Communities.

Could you please tell us which DevCloud you are using? Is it DevCloud for OneAPI/Edge/FPGA?

Regards,

Remya Premdas

[ffa]

oneapi

[ffa]

I want to reset both oneapi and edge devcloud. I have backup of my data. Please help with reset.

[RemyaP_Intel]

Hi, 

If you are using OneAPI environment, please make sure that you activated the OneAPI environment using the below command

source /opt/intel/inteloneapi/setvars.sh --force

Once the environment is activated, try these steps:

-2:~$ source /opt/intel/oneapi/setvars.sh



:: WARNING: setvars.sh has already been run. Skipping re-execution.

  To force a re-execution of setvars.sh, use the '--force' option.

  Using '--force' can result in excessive use of your environment variables.



usage: source setvars.sh [--force] [--config=file] [--help] [...]

 --force    Force setvars.sh to re-run, doing so may overload environment.

 --config=file Customize env vars using a setvars.sh configuration file.

 --help     Display this help message and exit.

 ...      Additional args are passed to individual env/vars.sh scripts

         and should follow this script's arguments.



 Some POSIX shells do not accept command-line options. In that case, you can pass

 command-line options via the SETVARS_ARGS environment variable. For example:



 $ SETVARS_ARGS="ia32 --config=config.txt" ; export SETVARS_ARGS

 $ . path/to/setvars.sh



 The SETVARS_ARGS environment variable is cleared on exiting setvars.sh.



@login-2:~$ qsub -I

qsub: waiting for job 2052767.v-qsvr-1.aidevcloud to start

qsub: job 2052767.v-qsvr-1.aidevcloud ready

########################################################################

#   Date:      Wed 23 Nov 2022 02:56:35 AM PST

#  Job ID:      2052767.v-qsvr-1.aidevcloud

# Resources:      neednodes=1:batch:ppn=2,nodes=1:batch:ppn=2,walltime=06:00:00

########################################################################

@s001-n036:~$ conda env list

# conda environments:

#

base         * /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest

2022.2.1         /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/2022.2.1

modin          /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/modin

modin-0.13.3       /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/modin-0.13.3

pytorch         /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/pytorch

pytorch-1.12.0      /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/pytorch-1.12.0

tensorflow        /glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/tensorflow

envtest         /home/u154867/.conda/envs/envtest

sample          /home/u154867/.conda/envs/sample


@s001-n036:~$ conda activate pytorch

(pytorch) @s001-n036:~$ which python

/glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/pytorch/bin/python

@s001-n036:~$ python

Python 3.9.13 (main, Oct 19 2022, 17:23:07)

[GCC 11.2.0] :: Intel Corporation on linux

Type "help", "copyright", "credits" or "license" for more information.

Intel(R) Distribution for Python is brought to you by Intel Corporation.

Please check out: https://software.intel.com/en-us/python-distribution

>>> import torch

>>>

Please try this first. OneAPI AI Toolkit already provides PyTorch and TensorFlow based conda environments and that might solve your problem.

Let us know if that doesn't solve your issue. We will check with the admin team if it is possible to reset DevCloud account.

Regards,

Remya Premdas

[ffa]

u177925@s001-n014:~$ conda activate pytorch
(pytorch) u177925@s001-n014:~$ which python
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/pytorch/bin/python
(pytorch) u177925@s001-n014:~$ python
Python 3.9.13 (main, Oct 19 2022, 17:23:07) 
[GCC 11.2.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
/home/u177925/.local/lib/python3.9/site-packages/torch/package/_directory_reader.py:17: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at /home/guizili/zhongruijie/frameworks.ai.pytorch.private-gpu/torch/csrc/utils/tensor_numpy.cpp:68.)
_dtype_to_storage = {data_type(0).dtype: data_type for data_type in _storages}
>>> import intel_extension_for_pytorch as ipex
>>> torch.xpu.is_available()
False
>>> 

I followed the steps and it seems it is working but I am getting a weird warning 

/home/u177925/.local/lib/python3.9/site-packages/torch/package/_directory_reader.py:17: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at /home/guizili/zhongruijie/frameworks.ai.pytorch.private-gpu/torch/csrc/utils/tensor_numpy.cpp:68.)
_dtype_to_storage = {data_type(0).dtype: data_type for data_type in _storages}

Can you please help with this?

[RemyaP_Intel]

Hi,

This shouldn't be a problem as it's just a warning. However, could you please try connecting to a different node and see if you are getting the same? Also, could you please try to print/find out the version of Numpy?

Regards,

Remya Premdas

[ffa]

u177925@s001-n064:~$ conda activate pytorch
(pytorch) u177925@s001-n064:~$ which python
/glob/development-tools/versions/oneapi/2022.3.1/oneapi/intelpython/latest/envs/pytorch/bin/python
(pytorch) u177925@s001-n064:~$ python
Python 3.9.13 (main, Oct 19 2022, 17:23:07) 
[GCC 11.2.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
/home/u177925/.local/lib/python3.9/site-packages/torch/package/_directory_reader.py:17: UserWarning: Failed to initialize NumPy: module compiled against API version 0x10 but this version of numpy is 0xe (Triggered internally at /home/guizili/zhongruijie/frameworks.ai.pytorch.private-gpu/torch/csrc/utils/tensor_numpy.cpp:68.)
_dtype_to_storage = {data_type(0).dtype: data_type for data_type in _storages}
>>> import numpy
>>> numpy.__version__
'1.21.4'
>>> torch.__version__
'1.10.0a0+git3d5f2d4'
>>> 

[RemyaP_Intel]

Hi,

Could you please manually create a separate virtual environment with the latest torch version and see if the warning is still coming? Otherwise, you may ignore the warning and proceed.

The torch version we have is different. It is v1.12.

Regards,

Remya Premdas

[ffa]

The torch version is the intel extension for pytorch one

[RemyaP_Intel · Accepted solution]

Hi,

The warning is because of the torch version 1.10.0. We created a separate virtual environment with torch=1.10.0 and got the same warning.

You can create a new virtual environment for torch version=1.12.1. This version of the torch will not give any warnings. Sharing the steps below for the same. Please try and let us know if you need any further clarification.

@login-2:~$ qsub -I
qsub: waiting for job 2077950.v-qsvr-1.aidevcloud to start
qsub: job 2077950.v-qsvr-1.aidevcloud ready

########################################################################
#      Date:           Thu 08 Dec 2022 02:02:47 AM PST
#    Job ID:           2077950.v-qsvr-1.aidevcloud
#      User:           
# Resources:           cput=35:00:00,neednodes=1:batch:ppn=2,nodes=1:batch:ppn=2,walltime=06:00:00
########################################################################

@s001-n027:~$
@s001-n027:~$ conda create -n  test_torch python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/uxxxxxx/.conda/envs/test_torch

  added / updated specs:
    - python=3.9


The following NEW packages will be INSTALLED:

  _libgcc_mutex      intel/linux-64::_libgcc_mutex-0.1-main
  _openmp_mutex      intel/linux-64::_openmp_mutex-4.5-1_gnu
  bzip2              intel/linux-64::bzip2-1.0.8-hb9a14ef_9
  ca-certificates    intel/linux-64::ca-certificates-2022.07.19-h06a4308_0
  certifi            intel/linux-64::certifi-2022.6.15-py39h06a4308_0
  intelpython        intel/linux-64::intelpython-2022.2.0-0
  libffi             intel/linux-64::libffi-3.3-14
  libgcc-ng          intel/linux-64::libgcc-ng-11.2.0-h1234567_1
  libgomp            intel/linux-64::libgomp-11.2.0-h1234567_1
  ncurses            intel/linux-64::ncurses-6.3-h5eee18b_3
  openssl            intel/linux-64::openssl-1.1.1q-h7f8727e_0
  pip                intel/linux-64::pip-22.1.2-py39h06a4308_0
  python             intel/linux-64::python-3.9.13-hb9903ff_6
  readline           intel/linux-64::readline-8.1.2-h7f8727e_1
  setuptools         intel/linux-64::setuptools-58.0.4-py39h06a4308_0
  sqlite             intel/linux-64::sqlite-3.39.2-h5082296_0
  tk                 intel/linux-64::tk-8.6.12-h1ccaba5_0
  wheel              intel/noarch::wheel-0.37.1-pyhd3eb1b0_0
  xz                 intel/linux-64::xz-5.2.5-h74280d8_2
  zlib               intel/linux-64::zlib-1.2.12-h5eee18b_3


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate test_torch
#
# To deactivate an active environment, use
#
#     $ conda deactivate

@s001-n027:~$ conda activate test_torch
(test_torch) @s001-n027:~$ conda install pytorch==1.12.1
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/uxxxxxx/.conda/envs/test_torch

  added / updated specs:
    - pytorch==1.12.1


The following NEW packages will be INSTALLED:

  blas               intel/linux-64::blas-1.0-mkl
  cffi               intel/linux-64::cffi-1.15.1-py39h74dc2b5_0
  future             intel/linux-64::future-0.18.2-py39h06a4308_1
  intel-openmp       intel/linux-64::intel-openmp-2021.4.0-intel_3561
  libstdcxx-ng       intel/linux-64::libstdcxx-ng-11.2.0-h1234567_1
  mkl                intel/linux-64::mkl-2021.4.0-intel_640
  mkl-service        pkgs/main/linux-64::mkl-service-2.4.0-py39h7f8727e_0
  mkl_fft            pkgs/main/linux-64::mkl_fft-1.3.1-py39hd3c417c_0
  mkl_random         pkgs/main/linux-64::mkl_random-1.2.2-py39h51133e4_0
  ninja              intel/linux-64::ninja-1.10.2-py39hff7bd54_0
  numpy              pkgs/main/linux-64::numpy-1.23.4-py39h14f4228_0
  numpy-base         pkgs/main/linux-64::numpy-base-1.23.4-py39h31eccc5_0
  pycparser          intel/noarch::pycparser-2.21-pyhd3eb1b0_0
  pytorch            pkgs/main/linux-64::pytorch-1.12.1-cpu_py39he8d8e81_0
  pyyaml             intel/linux-64::pyyaml-6.0-py39h7f8727e_1
  six                intel/noarch::six-1.16.0-pyhd3eb1b0_1
  tbb                intel/linux-64::tbb-2021.7.1-intel_15005
  typing-extensions  intel/noarch::typing-extensions-4.1.1-hd3eb1b0_0
  typing_extensions  intel/noarch::typing_extensions-4.1.1-pyh06a4308_0
  yaml               intel/linux-64::yaml-0.2.5-h7b6447c_0


Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(test_torch) @s001-n027:~$ python
Python 3.9.13 (main, Oct 19 2022, 17:23:07)
[GCC 11.2.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
torch>>> torch.__version__
'1.12.1'
>>>

Regards,

Remya Premdas

[RemyaP_Intel]

Hi,

Thanks for accepting our solution. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Regards,

Remya Premdas