Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1627 Discussions

Not able to change conda environment

PBhar
Student Ambassador
5,990 Views

When a user creates a new conda environment, he expects the environment to be be completely isolated from the global path. But in this case, the path remains fixed (/glob/intel-python/python3/bin//python)  and I'm not able to change.

I've attached the screenshot below.

0 Kudos
42 Replies
PBhar
Student Ambassador
1,939 Views

I tried it just now with the default environments.

qsub: waiting for job 557645.v-qsvr-1.aidevcloud to start
qsub: job 557645.v-qsvr-1.aidevcloud ready


########################################################################
#      Date:           Thu Mar 26 08:10:23 PDT 2020
#    Job ID:           557645.v-qsvr-1.aidevcloud
#      User:           u39403
# Resources:           mem=196gb,neednodes=4:ppn=2,nodes=4:ppn=2,walltime=24:00:00
########################################################################

(base) u39403@s001-n006:~$ conda info --envs
# conda environments:
#
base                  *  /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest
2021.1-beta05            /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/2021.1-beta05
pytorch                  /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/pytorch
pytorch-1.4.0            /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/pytorch-1.4.0
tensorflow               /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/tensorflow
tensorflow-1.15.0        /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/tensorflow-1.15.0
torch                    /home/u39403/.conda/envs/torch

(base) u39403@s001-n006:~$ conda activate pytorch
(pytorch) u39403@s001-n006:~$ python3
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
(pytorch) u39403@s001-n006:~$ conda activate pytorch-1.4.0
(pytorch-1.4.0) u39403@s001-n006:~$ python3
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
 

Not sure how is it working on your end. 

 

0 Kudos
Dan_P_Intel
Employee
1,939 Views

I have the same dilemma, not sure how it's not working on your end. ;)
Might be related to packages you've previously installed in your user space.

At this point I would look at a few options:

1. cleanup or refresh your side of things. You mentioned previously you installed some packages and perhaps there's a collision between various things.

To be fair I'm not an expert on Conda env management, so the following may or may not help:

https://docs.conda.io/projects/conda/en/latest/commands/clean.html
conda clean --yes --all

Remove custom environments:
conda remove --name myenv --all

etc. other related cleanup methods

2. You could give us permissions to enter your home folder and try and find a fix

3. We could try and reset your account to default but that may lead to lost files on your side.

 

Which option would you like to try?

0 Kudos
PBhar
Student Ambassador
1,939 Views

Yes, you have the permission. I'm tired of replying now. Please try it yourself and let me know if you figured out. 

0 Kudos
PBhar
Student Ambassador
1,939 Views

I have two accounts. The one I'm referring to has User Name: u39403. The other one also has the same problem but I use it primarily. 

0 Kudos
Dan_P_Intel
Employee
1,939 Views

No worries. I got that info from the terminal output you have posted previously. ;)

Would you mind if we simply backup your files and reset your account to factory settings? Might be faster than trying to debug what's going on.

0 Kudos
PBhar
Student Ambassador
1,939 Views

Please go ahead

0 Kudos
Dan_P_Intel
Employee
1,939 Views

Prajjwal, I've decided to wait a bit before going into your account.

Could you please try the following first?

1. mv ~/.conda ~/tmp/

if that does not help and there is pip involved, then

2. mv ~/.local ~/tmp/

Thank you!

0 Kudos
PBhar
Student Ambassador
1,939 Views

Done

0 Kudos
Dan_P_Intel
Employee
1,939 Views

Does Conda work now as you would expect it?

Thank you!

0 Kudos
PBhar
Student Ambassador
1,939 Views

I have not tried yet. I'll try using the instance and let you know any problems that I will encounter. 

0 Kudos
PBhar
Student Ambassador
1,939 Views

No it's not working. 

Please see below.

 

(base) u39403@login-2:~$ conda info --envs
WARNING: The conda.compat module is deprecated and will be removed in a future release.
# conda environments:
#
base                  *  /glob/intel-python/versions/2018u2/intelpython3

(base) u39403@login-2:~$ q
qsub: waiting for job 561084.v-qsvr-1.aidevcloud to start
qsub: job 561084.v-qsvr-1.aidevcloud ready


########################################################################
#      Date:           Tue Mar 31 06:45:03 PDT 2020
#    Job ID:           561084.v-qsvr-1.aidevcloud
#      User:           u39403
# Resources:           mem=196gb,neednodes=4:ppn=2,nodes=4:ppn=2,walltime=24:00:00
########################################################################

(base) u39403@s001-n055:~$ conda create --name torch
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/u39403/.conda/envs/torch

Proceed (/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
#     $ conda activate torch
#
# To deactivate an active environment, use
#
#     $ conda deactivate

(base) u39403@s001-n055:~$ conda activate torch
(torch) u39403@s001-n055:~$ conda install pytorch torchvision cpuonly -c pytorch
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/u39403/.conda/envs/torch

  added / updated specs:
    - cpuonly
    - pytorch
    - torchvision


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    cpuonly-1.0                |                0           2 KB  pytorch
    jpeg-9b                    |       h024ee3a_2         214 KB
    libtiff-4.1.0              |       h2733197_0         447 KB
    olefile-0.46               |           py37_0          50 KB
    pillow-7.0.0               |   py37hb39fc2d_0         598 KB
    pytorch-1.4.0              |      py3.7_cpu_0        39.0 MB  pytorch
    torchvision-0.5.0          |         py37_cpu        10.6 MB  pytorch
    zstd-1.3.7                 |       h0b5b093_0         401 KB
    ------------------------------------------------------------
                                           Total:        51.2 MB

The following NEW packages will be INSTALLED:

  bzip2              conda_channel/linux-64::bzip2-1.0.8-0
  certifi            conda_channel/linux-64::certifi-2019.11.28-py37_0
  cpuonly            pytorch/noarch::cpuonly-1.0-0
  freetype           conda_channel/linux-64::freetype-2.10.1-1
  icc_rt             conda_channel/linux-64::icc_rt-2021.1b5-intel_387
  intel-openmp       conda_channel/linux-64::intel-openmp-2021.1b5-intel_387
  intelpython        conda_channel/linux-64::intelpython-2021.1b5-0
  jpeg               pkgs/main/linux-64::jpeg-9b-h024ee3a_2
  libffi             conda_channel/linux-64::libffi-3.3-11
  libgcc-ng          conda_channel/linux-64::libgcc-ng-9.1.0-hdf63c60_0
  libpng             conda_channel/linux-64::libpng-1.6.37-2
  libstdcxx-ng       conda_channel/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0
  libtiff            pkgs/main/linux-64::libtiff-4.1.0-h2733197_0
  mkl                conda_channel/linux-64::mkl-2021.1b5-intel_336
  mkl-service        conda_channel/linux-64::mkl-service-2.3.0-py37_0
  mkl_fft            conda_channel/linux-64::mkl_fft-1.1.0-py37ha68da19_0
  mkl_random         conda_channel/linux-64::mkl_random-1.1.0-py37ha68da19_0
  ninja              conda_channel/linux-64::ninja-1.9.0-py37hfd86e86_0
  numpy              conda_channel/linux-64::numpy-1.17.5-py37ha68da19_1
  numpy-base         conda_channel/linux-64::numpy-base-1.17.5-py37_1
  olefile            pkgs/main/linux-64::olefile-0.46-py37_0
  openssl            conda_channel/linux-64::openssl-1.1.1d-0
  pillow             pkgs/main/linux-64::pillow-7.0.0-py37hb39fc2d_0
  pip                conda_channel/linux-64::pip-19.3.1-py37_2
  python             conda_channel/linux-64::python-3.7.5-hf484d3e_3
  pytorch            pytorch/linux-64::pytorch-1.4.0-py3.7_cpu_0
  setuptools         conda_channel/linux-64::setuptools-45.2.0-py37_0
  six                conda_channel/linux-64::six-1.13.0-py37_2
  sqlite             conda_channel/linux-64::sqlite-3.30.1-6
  tbb                conda_channel/linux-64::tbb-2021.1b5-intel_342
  tbb4py             conda_channel/linux-64::tbb4py-2021.1b5-py37_intel_0
  tcl                conda_channel/linux-64::tcl-8.6.9-24
  tk                 conda_channel/linux-64::tk-8.6.9-29
  torchvision        pytorch/linux-64::torchvision-0.5.0-py37_cpu
  wheel              conda_channel/linux-64::wheel-0.33.6-py37_4
  xz                 conda_channel/linux-64::xz-5.2.4-h14c3975_7
  zlib               conda_channel/linux-64::zlib-1.2.11.1-h14c3975_1
  zstd               pkgs/main/linux-64::zstd-1.3.7-h0b5b093_0


Proceed (/n)? y


Downloading and Extracting Packages
pillow-7.0.0         | 598 KB    | ############################################################################################################################################# | 100%
libtiff-4.1.0        | 447 KB    | ############################################################################################################################################# | 100%
cpuonly-1.0          | 2 KB      | ############################################################################################################################################# | 100%
olefile-0.46         | 50 KB     | ############################################################################################################################################# | 100%
torchvision-0.5.0    | 10.6 MB   | ############################################################################################################################################# | 100%
zstd-1.3.7           | 401 KB    | ############################################################################################################################################# | 100%
jpeg-9b              | 214 KB    | ############################################################################################################################################# | 100%
pytorch-1.4.0        | 39.0 MB   | ############################################################################################################################################# | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(torch) u39403@s001-n055:~$ python3
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
(torch) u39403@s001-n055:~$ which python
/glob/intel-python/python3/bin/python
(torch) u39403@s001-n055:~$
 

 

 

 

0 Kudos
Dan_P_Intel
Employee
1,939 Views

Prajjwal,

Have you looked at the following getting started guide?
https://devcloud.intel.com/oneapi/get-started/analytics-toolkit/

It covers our recommended way of using the PyTorch and TensorFlow versions that are installed through the Intel® AI Analytics Toolkit(Beta).

with apologies, could you please try the following steps:

1. mv ~/.conda ~/tmp/
2. mv ~/.local ~/tmp/

3. ssh devcloud
4. source /opt/intel/inteloneapi/setvars.sh
5. conda activate pytorch

The source .../setvars.sh command is required in order for this to work. We have added this command to the ~/.bash_profile of all (or at least most) users a while back but it seems that in your case that may not be true.

The AI DevCloud is being retired, I believe today is the last day. The old way of doing things on the DevCloud is not supported anymore.

0 Kudos
PBhar
Student Ambassador
1,939 Views

Please check your recommended solution before you inform me. You already have access to my account. 

 

(base) u39403@s001-n055:~$ source /opt/intel/inteloneapi/setvars.sh
:: setvars has already been run. Skipping any further invocation.  To force its re-execution, pass --force
(base) u39403@s001-n055:~$  conda activate pytorch
(pytorch) u39403@s001-n055:~$ python3
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
 

0 Kudos
Dan_P_Intel
Employee
1,939 Views

Good catch. Yes, you can't use the login node for your work.
You need to access a compute node.

Here's an updated list of instructions.

1. ssh devcloud
This will lead you to the login node.

2. mv ~/.conda ~/tmp/
3. mv ~/.local ~/tmp/
This was just to clean up your conda/pip environment.

4. qsub -I
Request an interractive session on a compute node.
Note: -I is upper case i not lower case L.

5. source /opt/intel/inteloneapi/setvars.sh
6. conda activate pytorch

The 4th step requests an interactive session on any compute node. This is great for debugging your projects but there's a better way if you want to execute some large workloads (for example ML training): the batch mode.

For more info about job submission see this link: https://devcloud.intel.com/oneapi/learn/job-submission/.

0 Kudos
PBhar
Student Ambassador
1,939 Views

Please carefully see what I posted. I tried your method, it doesn't work. I never asked about compute/login node. That's clearly evident from the snippet I shared. I'm again posting it here. 

 

(base) u39403@s001-n055:~$ source /opt/intel/inteloneapi/setvars.sh
:: setvars has already been run. Skipping any further invocation.  To force its re-execution, pass --force
(base) u39403@s001-n055:~$  conda activate pytorch
(pytorch) u39403@s001-n055:~$ python3
Python 3.6.3 |Intel Corporation| (default, Feb 12 2018, 06:37:09)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-15)] on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> import torch
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>

 

Pytorch is not imported. This should not happen.

0 Kudos
Dan_P_Intel
Employee
1,939 Views

Somehow you're starting the system python not the intelpython from oneAPI despite of sourcing the setvars.sh. I can only assume that certain changes on your profile are hiding the intelpython. Any thoughts?

What would be a good time window for us to go in using your account? I'd like to avoid interfering with your work.

0 Kudos
PBhar
Student Ambassador
1,939 Views

Please go ahead (log in to my account). I'm not using it.

0 Kudos
PBhar
Student Ambassador
1,939 Views

Can you please print exact steps on how a user is supposed to create a fresh conda environment and install packages ? Please test it out properly. Whatever I receive on these forums are either half cooked solutions or they don't work at all.

0 Kudos
PBhar
Student Ambassador
1,939 Views

It seems that users can't install any packages now in intel provided environments.  They're greeted with this:


EnvironmentNotWritableError: The current user does not have write permissions to the target environment.
  environment location: /glob/development-tools/versions/oneapi/beta05/inteloneapi/intelpython/latest/envs/pytorch-1.4.0

 

Why are there so many restrictions ? Why can't a user do a fresh install by themselves ? I think docker support would be more handy rather than using what is provided.

0 Kudos
Dan_P_Intel
Employee
1,704 Views

Prajjwal, thank you for your patience.

I have accessed your account and found a conda initialization section in your .bashrc config file. This section configured the python in /glob/intel-python as your default python, a valid setting during the lifetime of AI DevCloud. Since the AI DevCloud was retired on 03/31/2020, the initialization section is not necessary anymore, at least in the current form.

In order to do some testing, I have commented out that initialization section and this time the steps I mentioned in a previous post passed. Please try it and let me know if that addresses your issue.

Based on your experience we're conducting additional investigations & fixes. One related issue we found is that after conda deactivate the oneAPI environment variables might be disabled. Our recommendation is to explicitly run "source /opt/intel/inteloneapi/setvars.sh" before any meaningful work.

As a thank you for your continuing efforts to improve the Intel(R) DevCloud we've extended your access by one additional year. Thank you!
 

0 Kudos
AthiraM_Intel
Moderator
1,704 Views

Hi,

Could you please confirm, whether the steps mentioned previously are working fine now?

Thanks

0 Kudos
Reply