topic Re: oneAPI AI Toolkit in AI Tools from Intel

oneAPI AI Toolkit

StephB — Fri, 13 May 2022 15:42:49 GMT

I have installed oneAPI off-line successfully on Ubuntu 20.04. I am trying my hand at my first Neural Network.

I do the setvars.sh in /etc/profile.d/ (i.e. source /opt/intel/oneapi/setvars.sh) so that it is available for all users on my system.

I follow a course on Youtube and the first step is to create data using a random number generator, using the following Python modules:

import numpy as np
from random import randint
from sklearn.utils import shuffle
from sklearn.preprocessing import MinMaxScaler

The generation of data works perfectly fine. Now the second step is to create the neural network and for that I must add the following modules:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import categorical_crossentropy

for this to work I must activate tensorflow as follows:

source activate tensorflow

Once I've done this I get the message

ModuleNotFoundError: No module named 'sklearn'

I have consulted the forums but couldn't find a solution or guidance yet, I'll appreciate some assistance.

Thank you

Re:oneAPI AI Toolkit

JananiC_Intel — Mon, 16 May 2022 11:47:08 GMT

Hi,

Thanks for posting in Intel forum.

Could you install scikit-learn in tensorflow environment with "conda install -c intel scikit-learn" command ?

Install this package and let us know if you get any issues.

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Mon, 16 May 2022 13:46:13 GMT

Please see the result below. Conda is available on my PC, but seems not in the tensorflow environment? I have tried sudo as well (intel is installed in opt).

(tensorflow) steph@xxxxx:$ conda install -c intel scikit-learn
Collecting package metadata (current_repodata.json): failed

NoWritablePkgsDirError: No writeable pkgs directories configured.
- /opt/intel/oneapi/intelpython/latest/pkgs
- /home/steph/.conda/pkgs

(tensorflow) steph@xxxxx:$

Thank you very much for assistance.

Re:oneAPI AI Toolkit

JananiC_Intel — Tue, 17 May 2022 11:03:00 GMT

Hi,

Thanks for the update.

You are using default AI virtual environment in which you cannot install additional package. So you can try cloning this default tensorflow environment into a new custom environment with the below command.

conda create --name myclone --clone tensorflow

where myclone is new custom environment.

After this you can activate myclone environment and install scikit-learn in that new custom environment with the below command.

conda install -c intel scikit-learn

Try this and let us know if you have any issues.

Regards,

Janani Chandran

Re:oneAPI AI Toolkit

JananiC_Intel — Tue, 24 May 2022 10:42:53 GMT

Hi,

Is your issue resolved? Do you have any update?

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Tue, 24 May 2022 15:12:17 GMT

Dear Janani

Thank you very much for caring. I am not sure that I understand the proposed solution, so I have decided to try the SKlearn package first. I have now successfully created a model with SKlearn. It is now time to move on to Tensorflow again. I plan to invite a friend with more computer background to assist me with your proposed solution.

Once again thanks for caring.

Regards

Re: oneAPI AI Toolkit

StephB — Thu, 26 May 2022 09:17:15 GMT

Dear Janani

I am trying to understand, must the cloning of the default tensorflow environment into a new custom environment be done as sudo, and where is this custom environment created? I am afraid to create something, and it is wrong and I cannot reverse the process again.

Kind regards

Re:oneAPI AI Toolkit

JananiC_Intel — Fri, 27 May 2022 04:29:46 GMT

Hi,

When you give the command "conda create --name myclone --clone tensorflow" , it will create a new environment and will clone the packages from default tensorflow environment to the new environment and this does not require sudo.

Also you can find the list of created environments with the command "conda env list". Once it is created, activate your environment with "conda activate <env_name>" or "source activate <env_name>" command.

Hope this answers your query.

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Sat, 28 May 2022 07:40:23 GMT

Thank you for once again assisting.

First, it seemed that Conda was not installed on my system, so I installed it. I downloaded the latest Conda and installed it with "bash Anaconda3-2022.05-Linux-x86_64.sh".

I have issued the command "conda create --name myclone --clone tensorflow" (I have decided to use myclone to start with, I might come up with a more creative name later on).

This was the message:

EnvironmentNameNotFound: Could not find conda environment: tensorflow
You can list all discoverable environments with `conda info --envs`.

So, I listed the known environments:

(base) steph@xxxxx:$ conda info --envs
# conda environments:
#
base * /home/steph/anaconda3

Conda installation added the following to my ~.bashrc file

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/steph/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/steph/anaconda3/etc/profile.d/conda.sh" ]; then
. "/home/steph/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/home/steph/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

On my system Intel API is activated in /etc/profile.d/intel_api_vars.sh and this file contains two lines:

#prepare for Intel compilers
source /opt/intel/oneapi/setvars.sh

I trust this will provide more information on my setup. I have not tried sudo.. yet.

Thanks.

Re:oneAPI AI Toolkit

JananiC_Intel — Tue, 31 May 2022 13:19:52 GMT

Hi,

Thanks for detailed explanation.

For your information, if oneAPI AI toolkit is installed successfully the default tensorflow environment comes with that . In order to confirm the presence of tensorflow environment use the below command.

find -iname tensorflow

Usually it will be in "oneapi/2022.2/oneapi/intelpython/latest/envs/tensorflow" path.

Try the above command and share the results.

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Wed, 01 Jun 2022 08:28:16 GMT

Thanks for this. find yields a lot "permission denied" results as oneAPI is installed in opt, see below:

/opt/intel/oneapi/intelpython/latest/envs/tensorflow

/opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.8.0

And I have added the following now: I use Spyder as my editor so there is a Spyder environment, see below. The ~\.coda\environments.txt file belonged to root so I have chown to steph.

/home/steph/anaconda3/envs/spyder-env

Trust this will help.

Re:oneAPI AI Toolkit

JananiC_Intel — Fri, 03 Jun 2022 09:50:23 GMT

Hi,

Thanks for the update.

Kindly follow the below steps. Also, the below steps will not cause any permission denied issues.

1)Go to tensorflow directory

cd /opt/intel/oneapi/intelpython/latest/envs/tensorflow

2)Activate tensorflow environment with the below command.

source vars.sh

After this 2nd step, your environment will get activated and you can proceed with the cloning steps which was mentioned earlier in this thread.

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Wed, 08 Jun 2022 20:05:35 GMT

There is no vars.sh file in this directory? What I do see is the following in the env directory:

steph@xxxxx:$ ls -l env/
total 24
-rwxrwxr-x 5 root root 10318 Mar 29 13:09 compiler_rt_vars.sh
-rw-rw-r-- 5 root root 11929 Jan 12 19:31 vars.sh
steph@xxxxx:$

Must I execute this file?

Re: oneAPI AI Toolkit

JananiC_Intel — Thu, 09 Jun 2022 06:55:43 GMT

Hi,

Thanks for the update.

Did you search in "/opt/intel/oneapi/intelpython/latest/envs/tensorflow/env" path ? Can you confirm?

In one of your previous posts, you have mentioned that you have tensorflow in "/opt/intel/oneapi/intelpython/latest/envs/tensorflow" path. If so, just give "ls" command from env directory and share the result. Normally there will be compiler_rt_vars.sh and vars.sh in env directory of tensorflow and once we source vars.sh, tensorflow environment will get activated . Refer the attached screenshot for reference.

Regards,

Janani Chandran

Re: oneAPI AI Toolkit

StephB — Thu, 09 Jun 2022 19:52:29 GMT

I have shown the ls in env in my previous with results as follows

steph@xxxxx:$ ls -l env/
total 24
-rwxrwxr-x 5 root root 10318 Mar 29 13:09 compiler_rt_vars.sh
-rw-rw-r-- 5 root root 11929 Jan 12 19:31 vars.sh
steph@xxxxx:$

I wanted confirmation please that that is the file I must execute. I think have now confirmed that that is what I must do, I'll continue. Thanks again.

I have doen everything, seems I am sorted! Now I think I must install Spyder again in this new environment.

Edit:

I have installed Spyder and everything seems in order.

Many thanks again

Re:oneAPI AI Toolkit

JananiC_Intel — Fri, 10 Jun 2022 10:17:49 GMT

Hi,

Glad to know that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Regards,

Janani Chandran