Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud

Python 3?

couto__carlos
Beginner
1,387 Views

Hi

 

I've been an user of intel devcloud since long. I was able to submit and run my scripts without any problem. For some reason this is not working anymore (last time I used and working was 1 or 2 months ago).

 

What I see is that default python is now Python2 (?)  and packages such as sklearn is no longer available. Can you please let me know how to proceed?

 

Thanks.

0 Kudos
1 Solution
mike02
Novice
1,064 Views

[Edited]

I made some progress since my last reply.  To summarize I have a work-around hack  that allows the  "Intel DevCloud for oneAPI Projects"  (what @couto__carlos terms connecting via ssh) environment to be used. Found this by working backwards from the  jupyterlab notebook environment

 

As we saw before after logging in via ssh  you see the wrong version of python 

> python

python 2.7.17 (default, Nov 7 2019, 10:07:09)
[GCC 7.4.0] on linux2

 

Here is a work around - You can force it to use the same python executable as jupyterlab by adding this line to the end of your ~/.bashrc

 

 

export PATH='/glob/development-tools/versions/oneapi/2024.1/oneapi/intelpython/envs/pytorch/bin/:'${PATH}

 

 

reload your shell

 

source ~/.bashrc

 

 

Now you will get the same python version used by jupyterlab

 

> python 

Python 3.9.18 (tags/v3.9.18-26-g6b320c3b2f6-dirty:6b320c3b2f6, Sep 28 2023, 00:35:27)
[GCC 13.2.0] :: Intel Corporation on linux
(null)Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> 

 

pip now work as expected

 

>> pip list | grep -i torch
intel-extension-for-pytorch 2.2.0+cpu
pytorchvideo                0.1.5
torch                       2.2.0+cpu
torchaudio                  2.2.0+cpu
torchvision                 0.17.1+4fd856b

 

 

 

Note that to really be able to use the environment you need to log into an interactive node using something like (see https://devcloud.intel.com/oneapi/documentation/job-submission

 

qsub -I -l nodes=1:gpu:ppn=2 -d .

 

 

View solution in original post

8 Replies
mike02
Novice
1,266 Views

Does anyone have a solution to the question? I am experiencing the same issue. The last time I used devcloud was in January 2024. when python was working fine. Now I also see python version is 2.7.17

 

Simple commands now also give errors for example

pip list

-bash: /glob/intel-python/python2/bin/pip: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory

 

conda --help

-bash: /glob/intel-python/versions/2018u2/intelpython2/bin/conda: /glob/intel-python/versions/2018u2/intelpython2/bin/python: bad interpreter: No such file or directory

 

Does anyone know what may have gone wrong with the environment

 

 

 

0 Kudos
Mark_Rosenbaum
New Contributor I
1,173 Views

Which DevCloud environment are you experiencing this issue within? (ex. Edge, oneAPI)

0 Kudos
Mark_Rosenbaum
New Contributor I
1,168 Views

Within the Edge JupyterLab environment I'm unable to reproduce the issue.(see attached screenshot)

0 Kudos
mike02
Novice
1,131 Views

Thanks for your observation. I am using Intel DevCloud for oneAPI Projects (It is a command line interface)

 

I also verified that these issues do not occur using the jupyterLab environment. The nice thing there is that one can pick and change the python kernel

 

I forced the oneApi for projects to use one of the python versions I used in jupyterLab (for example pytorch 2024.1) 

It sort of works until I try import all the packages I need (opencv, torch, etc) where I start getting errors, for example

>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/glob/development-tools/versions/oneapi/2023.2.0.1/inteloneapi/pytorch/latest/lib/python3.9/site-packages/torch/__init__.py", line 229, in <module>
from torch._C import * # noqa: F403
ImportError: libtorch_cpu.so: failed to map segment from shared object

 

 

I think this generally means a mismatch between versions of the packages used when building torch

 

 

 

0 Kudos
couto__carlos
Beginner
1,098 Views

It is still not working. Attached is the "Terminal" launched from JupyterLab as well as connecting via ssh. Notice that via ssh this was working until recently when I was requested to extend my access.

 

*edit: some corrections to original reply.

0 Kudos
mike02
Novice
1,065 Views

[Edited]

I made some progress since my last reply.  To summarize I have a work-around hack  that allows the  "Intel DevCloud for oneAPI Projects"  (what @couto__carlos terms connecting via ssh) environment to be used. Found this by working backwards from the  jupyterlab notebook environment

 

As we saw before after logging in via ssh  you see the wrong version of python 

> python

python 2.7.17 (default, Nov 7 2019, 10:07:09)
[GCC 7.4.0] on linux2

 

Here is a work around - You can force it to use the same python executable as jupyterlab by adding this line to the end of your ~/.bashrc

 

 

export PATH='/glob/development-tools/versions/oneapi/2024.1/oneapi/intelpython/envs/pytorch/bin/:'${PATH}

 

 

reload your shell

 

source ~/.bashrc

 

 

Now you will get the same python version used by jupyterlab

 

> python 

Python 3.9.18 (tags/v3.9.18-26-g6b320c3b2f6-dirty:6b320c3b2f6, Sep 28 2023, 00:35:27)
[GCC 13.2.0] :: Intel Corporation on linux
(null)Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
>>> 

 

pip now work as expected

 

>> pip list | grep -i torch
intel-extension-for-pytorch 2.2.0+cpu
pytorchvideo                0.1.5
torch                       2.2.0+cpu
torchaudio                  2.2.0+cpu
torchvision                 0.17.1+4fd856b

 

 

 

Note that to really be able to use the environment you need to log into an interactive node using something like (see https://devcloud.intel.com/oneapi/documentation/job-submission

 

qsub -I -l nodes=1:gpu:ppn=2 -d .

 

 

Mark_Rosenbaum
New Contributor I
1,040 Views

@couto__carlos I was able to reproduce your issue within the environment you specified and I was also able to test @mike02's solution, it works perfectly.  While it would be nice to just have Intel fix the issue instead of using such a hacky solution whatever works works.  If you test his solution and it works for you I suggest accepting his solution so that others see it in the future.

-Mark

0 Kudos
couto__carlos
Beginner
751 Views

Many thanks to @mike02 and @Mark_Rosenbaum for your replies. This is working following the solution @mike02 . I am marking this as solved. It is likely that some problems still exist as mentioned using pytorch in the previous replies.

0 Kudos
Reply