intel-oneapi-pytorch repo unreachable from apt-get

[KevinR_Intel]

I am trying to install oneAPI tools on a fresh Ubuntu machine. When I try
sudo apt-get install -y intel-aikit
I get the error

After this operation, 2479 MB of additional disk space will be used.
Err:1 https://apt.repos.intel.com/oneapi all/main all intel-oneapi-pytorch amd64 1.13.10.1-47264
404 Not Found [IP: 69.192.80.228 443]
E: Failed to fetch https://apt.repos.intel.com/oneapi/pool/main/intel-oneapi-pytorch-1.13.10.1-47264_amd64.deb 404 Not Found [IP: 69.192.80.228 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

This procedure for installing was working about 1 week ago. Did something happen to break a single repo?

[whatheway]

workhorse@workhorse:~/WRF-MOSIT$ sudo apt -y install intel-aikit

Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
intel-aikit-getting-started intel-oneapi-common-licensing-2023.0.0
intel-oneapi-model-zoo intel-oneapi-modin intel-oneapi-mpi-2021.8.0
intel-oneapi-mpi-devel-2021.8.0 intel-oneapi-neural-compressor
intel-oneapi-pytorch intel-oneapi-tensorflow
The following NEW packages will be installed:
intel-aikit intel-aikit-getting-started
intel-oneapi-common-licensing-2023.0.0 intel-oneapi-model-zoo
intel-oneapi-modin intel-oneapi-mpi-2021.8.0 intel-oneapi-mpi-devel-2021.8.0
intel-oneapi-neural-compressor intel-oneapi-pytorch intel-oneapi-tensorflow
0 upgraded, 10 newly installed, 0 to remove and 2 not upgraded.
Need to get 462 MB/1,784 MB of archives.
After this operation, 2,197 MB of additional disk space will be used.

Err:1 https://apt.repos.intel.com/oneapi all/main all intel-oneapi-pytorch amd64 1.13.10.1-47264
404 Not Found [IP: 104.69.89.197 443]

E: Failed to fetch https://apt.repos.intel.com/oneapi/pool/main/intel-oneapi-pytorch-1.13.10.1-47264_amd64.deb 404 Not Found [IP: 104.69.89.197 443]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[KevinR_Intel]

sudo apt-get install -y intel-oneapi-pytorch
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
intel-oneapi-dal-daal4py-2023.1.0 intel-oneapi-dal-scikit-learn-intelex-2023.1.0 intel-oneapi-python
The following NEW packages will be installed:
intel-oneapi-dal-daal4py-2023.1.0 intel-oneapi-dal-scikit-learn-intelex-2023.1.0 intel-oneapi-python intel-oneapi-pytorch
0 upgraded, 4 newly installed, 0 to remove and 60 not upgraded.
Need to get 462 MB/743 MB of archives.
After this operation, 746 MB of additional disk space will be used.
Err:1 https://apt.repos.intel.com/oneapi all/main all intel-oneapi-pytorch amd64 1.13.10.1-47264
404 Not Found [IP: 23.62.86.197 443]
E: Failed to fetch https://apt.repos.intel.com/oneapi/pool/main/intel-oneapi-pytorch-1.13.10.1-47264_amd64.deb 404 Not Found [IP: 23.62.86.197 443]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[whatheway]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing

So this suggestion from Intel does not fix the problem.

[KevinR_Intel]

That's actually the preformed suggestion from apt-get; it has the potential to solve some issues. I was adding my error result with a slightly different command, and that's the last line from the output. Sorry for the misunderstanding.

[whatheway]

No worries,

There still is the 404 missing link so I guess we have to wait till the admins and software developers fix it?

[ArpanB_Intel]

Hi Kevin, we are currently working to fix this issue as early as possible. Meanwhile, we appreciate your patience.

[Aditya18]

Hi @Kevin,

I could not see the issue using these instructions from https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit-download.html?operatingsystem=linux&distributions=aptpackagemanager

wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB \

| gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null

echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list

sudo apt update

sudo apt install intel-aikit

Please check these. And let us know if there is any issue.

[whatheway]

@Aditya18 

I can confirm that this does not work on Ubuntu 22.04.

[whatheway]

@Aditya18 @KevinR_Intel @ArpanB_Intel 

The current stable version of pytorch is 1.13.0.0 installed from the offline installer

[kta_intel]

@KevinR_Intel  @whatheway 

Hi,

We tried running the instructions from https://www.intel.com/content/www/us/en/developer/tools/oneapi/ai-analytics-toolkit-download.html?operatingsystem=linux&distributions=aptpackagemanager as @Aditya18 mentioned on a fresh Ubuntu 22.04 instance and were not able to reproduce the issues (i.e. we did not run into any installation issues).

See pytorch conda env:

Packages fetched:

Can you try reinstalling using the provided steps, including the prerequisites steps? Thanks

[Aditya18]

Hi @KevinR_Intel ,

In Ubuntu 22.04, I couldn't see any error logs you posted that....

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

And the PyTorch version showed in your logs is little old... 1.13.10.1 But when we installed it's 1.13.10.2

Though I saw this in the end,

Pytorch installation file doesn't exist. Cannot install Pytorch and Neural-Compressor.

Setting up intel-oneapi-pytorch (1.13.10.2-48866) ...

Setting up intel-oneapi-modin (0.19.0.0-48867) ...

Setting up intel-aikit (2023.1.1-48862) ...

But when I checked all the env after activate oneAPI, the following PyTorch env is present(the same Kevin Ta shared) and also working.

pytorch-1.13.10.2    /opt/intel/oneapi/intelpython/latest/envs/pytorch-1.13.10.2

[KevinR_Intel]

Starting over from a fresh Ubuntu 22, the procedure seems to proceed (with some error messages). The important progress is that the 404 error has been fixed.

But something in the meta-package install is still not quite correct:  The following directory is not created
/opt/intel/oneapi/intelpython/python3.9/bin

and it causes a cascade of step failuires.

ONEAPI_PATH:/opt/intel/oneapi/tensorflow/2.12.0/../..
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 34: ../../intelpython/python3.9/bin/conda: not found
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 37: ../../intelpython/python3.9/bin/conda: not found
cp: cannot stat '../../intelpython/python3.9/envs/tensorflow-gpu/share/intel_extension_for_tensorflow/libstdc/activate-libstdcxx-patch.sh': No such file or directory
cp: cannot stat '../../intelpython/python3.9/envs/tensorflow-gpu/share/intel_extension_for_tensorflow/libstdc/deactivate-libstdcxx-patch.sh': No such file or directory
chmod: cannot access '../../intelpython/python3.9/pkgs/*/*': No such file or directory
ONEAPI_PATH:/opt/intel/oneapi/neural-compressor/2.1.0/../..
Framework parameter is set to: tensorflow
Neural Compressor will be installed in tensorflow environment
Installing Neural Compressor in tensorflow environment
/opt/intel/oneapi/tensorflow/2.12.0/../../neural-compressor/latest/install_neural-compressor.sh: 52: .: cannot open /opt/intel/oneapi/neural-compressor/2.1.0/../../intelpython/latest/bin/activate: No such file
ONEAPI_PATH:/opt/intel/oneapi/neural-compressor/2.1.0/../..
Framework parameter is set to: tensorflow-gpu
Neural Compressor will be installed in tensorflow-gpu environment
Installing Neural Compressor in tensorflow-gpu environment
Installation failed, tensorflow-gpu environment not found. Please check tensorflow-gpu installation script.
tensorflow-gpu install script location: /opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow-gpu/latest/install_tensorflow-gpu.sh
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 61: ../../intelpython/python3.9/bin/conda: not found
/
Pytorch installation file doesn't exist. Cannot install Pytorch and Neural-Compressor.
Setting up intel-oneapi-pytorch (1.13.10.2-48866) ...
Setting up intel-oneapi-modin (0.19.0.0-48867) ...
Setting up intel-aikit (2023.1.1-48862) ...
Scanning processes...
Scanning linux images...

[kta_intel]

Hmm, I'm still unable to reproduce this issue. Just to cover our bases, are you getting any issues or unexpected results when running the prerequisite commands?

Also, are you able to install the AI kit through the other installation methods without issue? (And if so, I'm assuming this isn't an acceptable workaround for you?)

It's possible we may need to escalate if the issue is now related to the creation of /opt/intel/oneapi/intelpython/python3.9/bin as not a missing pytorch package, but we can try to troubleshoot some more. Wonder if it's a systems requirement issue. Can you just double check that you have the min system requirements? https://www.intel.com/content/www/us/en/developer/articles/system-requirements/intel-oneapi-ai-analytics-toolkit-system-requirements.html

[Aditya18]

Hi @KevinR_Intel

Yes, as far as I remember, even I saw some of those last few lines of logs. And I was able to see the pytorch env and other envs. We can look into that directory not created.

Before that, could you activate oneAPI using "source /opt/intel/oneapi/setvars.sh" and check which envs are there... "conda env list"? Wanted to have a look at the envs you got and their paths.

[KevinR_Intel]

I began with a fresh Ubuntu 22.04 LTS

Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-56-generic x86_64)

The first command was a typo, but these are all of the commands executed

ubuntu@cloud36970:~$ history
1 sudo apt get update
2 sudo apt-get update
3 sudo apt-get upgrade
4 wget -O- https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB | gpg --dearmor | sudo tee /usr/share/keyrings/oneapi-archive-keyring.gpg > /dev/null
5 echo "deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main" | sudo tee /etc/apt/sources.list.d/oneAPI.list
6 sudo apt update
7 sudo apt install intel-aikit
8 source /opt/intel/oneapi/setvars.sh

 During apt install intel-aikit, I observed

Setting up intel-oneapi-neural-compressor (2.1.0-48866) ...
ONEAPI_PATH:/opt/intel/oneapi/neural-compressor/2.1.0/../..
Framework parameter is set to: tensorflow,pytorch
TensorFlow will be installed. Neural Compressor installation will follow in TensorFlow environment.
ONEAPI_PATH:/opt/intel/oneapi/tensorflow/2.12.0/../..
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 34: ../../intelpython/python3.9/bin/conda: not found
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 37: ../../intelpython/python3.9/bin/conda: not found
cp: cannot stat '../../intelpython/python3.9/envs/tensorflow-gpu/share/intel_extension_for_tensorflow/libstdc/activate-libstdcxx-patch.sh': No such file or directory
cp: cannot stat '../../intelpython/python3.9/envs/tensorflow-gpu/share/intel_extension_for_tensorflow/libstdc/deactivate-libstdcxx-patch.sh': No such file or directory
chmod: cannot access '../../intelpython/python3.9/pkgs/*/*': No such file or directory
ONEAPI_PATH:/opt/intel/oneapi/neural-compressor/2.1.0/../..
Framework parameter is set to: tensorflow
Neural Compressor will be installed in tensorflow environment
Installing Neural Compressor in tensorflow environment
/opt/intel/oneapi/tensorflow/2.12.0/../../neural-compressor/latest/install_neural-compressor.sh: 52: .: cannot open /opt/intel/oneapi/neural-compressor/2.1.0/../../intelpython/latest/bin/activate: No such file
ONEAPI_PATH:/opt/intel/oneapi/neural-compressor/2.1.0/../..
Framework parameter is set to: tensorflow-gpu
Neural Compressor will be installed in tensorflow-gpu environment
Installing Neural Compressor in tensorflow-gpu environment
Installation failed, tensorflow-gpu environment not found. Please check tensorflow-gpu installation script.
tensorflow-gpu install script location: /opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow-gpu/latest/install_tensorflow-gpu.sh
/opt/intel/oneapi/neural-compressor/2.1.0/../../tensorflow/latest/install_tensorflow.sh: 61: ../../intelpython/python3.9/bin/conda: not found
/
Pytorch installation file doesn't exist. Cannot install Pytorch and Neural-Compressor.

 The remaining 'Setting up ...' steps proceed without complaint. The result of using the 'source' instruction,

ubuntu@cloud36970:~$ source /opt/intel/oneapi/setvars.sh

:: initializing oneAPI environment ...
-bash: BASH_VERSION = 5.1.16(1)-release
args: Using "$@" for setvars.sh arguments:
:: compiler -- latest
:: dal -- latest
:: dev-utilities -- latest
:: ipp -- latest
:: mkl -- latest
:: modelzoo -- latest
:: mpi -- latest
:: neural-compressor -- latest
:: pytorch -- latest
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
:: tbb -- latest
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
:: oneAPI environment initialized ::

And when I try the conda instruction you suggest

ubuntu@cloud36970:~$ conda env list
ERROR: ld.so: object '/usr/lib64/libstdc++.so.6.0.30.' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
conda: command not found

I'm under the impression that the meta-package intel-aikit grabs all the components. Did I miss some other instruction to install the intelpython?

[kta_intel]

The steps seem fine, your impression is correct and intelpython should've been installed automatically. Are you using a devcloud beta instance? If so, which one are you using? I'll try to see if I can reproduce on that instead of an azure instance

[KevinR_Intel]

Yes, I am; it's one of the "Large Virtual Machine" instances.

[kta_intel]

Apologies for the delay. Thanks for the info, your issue is reproducible on the Large Virtual Machine instance. We're investigating it now

[kta_intel]

It looks like I made a mistake in a previous reply. The azure instance I tested on a while back was actually 20.04... I just spun up a 22.04 instance to double check, and I've indeed been able to reproduced your issue there as well. Looking at the system requirements, it seems as though the AI kit does not yet support 22.04

I will try to confirm this and, if this is the case, create a request to upgrade the support

@Aditya18 You mentioned that you were able to activate the AI kit environments in Ubuntu 22.04, can you confirm? I just want to capture everything we have for when I open the ticket

[susie]

Has this issue been solved? I ran into same problem on Fedora 35. Thanks