oneAPI AI Toolkit

[StephB]

I have installed oneAPI off-line successfully on Ubuntu 20.04. I am trying my hand at my first Neural Network.

I do the setvars.sh in /etc/profile.d/ (i.e. source /opt/intel/oneapi/setvars.sh) so that it is available for all users on my system.

I follow a course on Youtube and the first step is to create data using a random number generator, using the following Python modules:

import numpy as np
from random import randint
from sklearn.utils import shuffle
from sklearn.preprocessing import MinMaxScaler

The generation of data works perfectly fine. Now the second step is to create the neural network and for that I must add the following modules:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Activation, Dense
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.metrics import categorical_crossentropy

for this to work I must activate tensorflow as follows:

source activate tensorflow

Once I've done this I get the message 

ModuleNotFoundError: No module named 'sklearn'

I have consulted the forums but couldn't find a solution or guidance yet, I'll appreciate some assistance.

Thank you

[JananiC_Intel]

Hi,

Thanks for posting in Intel forum.

Could you install scikit-learn in tensorflow environment with "conda install -c intel scikit-learn" command ?

Install this package and let us know if you get any issues.

Regards,

Janani Chandran

[StephB]

Please see the result below. Conda is available on my PC, but seems not in the tensorflow environment? I have tried sudo as well (intel is installed in opt). 

(tensorflow) steph@xxxxx:$ conda install -c intel scikit-learn
Collecting package metadata (current_repodata.json): failed

NoWritablePkgsDirError: No writeable pkgs directories configured.
- /opt/intel/oneapi/intelpython/latest/pkgs
- /home/steph/.conda/pkgs

(tensorflow) steph@xxxxx:$

Thank you very much for assistance.

[JananiC_Intel]

Hi,

Thanks for the update.

You are using default AI virtual environment in which you cannot install additional package. So you can try cloning this default tensorflow environment into a new custom environment with the below command.

conda create --name myclone --clone tensorflow

where myclone is new custom environment.

After this you can activate myclone environment and install scikit-learn in that new custom environment with the below command.

conda install -c intel scikit-learn

Try this and let us know if you have any issues.

Regards,

Janani Chandran

[JananiC_Intel]

Hi,

Is your issue resolved? Do you have any update?

Regards,

Janani Chandran

[StephB]

Dear Janani

Thank you very much for caring. I am not sure that I understand the proposed solution, so I have decided to try the SKlearn package first. I have now successfully created a model with SKlearn. It is now time to move on to Tensorflow again. I plan to invite a friend with more computer background to assist me with your proposed solution.

Once again thanks for caring.

Regards

[StephB]

Dear Janani

I am trying to understand, must the cloning of the default tensorflow environment into a new custom environment be done as sudo, and where is this custom environment created? I am afraid to create something, and it is wrong and I cannot reverse the process again.

Kind regards

[JananiC_Intel]

Hi,

When you give the command "conda create --name myclone --clone tensorflow" , it will create a new environment and will clone the packages from default tensorflow environment to the new environment and this does not require sudo.

Also you can find the list of created environments with the command "conda env list". Once it is created, activate your environment with "conda activate <env_name>" or "source activate <env_name>" command.

Hope this answers your query.

Regards,

Janani Chandran

[StephB]

Thank you for once again assisting.

First, it seemed that Conda was not installed on my system, so I installed it. I downloaded the latest Conda and installed it with "bash Anaconda3-2022.05-Linux-x86_64.sh". 

I have issued the command "conda create --name myclone --clone tensorflow" (I have decided to use myclone to start with, I might come up with a more creative name later on).

This was the message:

EnvironmentNameNotFound: Could not find conda environment: tensorflow
You can list all discoverable environments with `conda info --envs`.

So, I listed the known environments:

(base) steph@xxxxx:$ conda info --envs
# conda environments:
#
base * /home/steph/anaconda3

Conda installation added the following to my ~.bashrc file

# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/steph/anaconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/steph/anaconda3/etc/profile.d/conda.sh" ]; then
. "/home/steph/anaconda3/etc/profile.d/conda.sh"
else
export PATH="/home/steph/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<

On my system Intel API is activated in /etc/profile.d/intel_api_vars.sh and this file contains two lines:

#prepare for Intel compilers
source /opt/intel/oneapi/setvars.sh

I trust this will provide more information on my setup. I have not tried sudo.. yet.

Thanks.

[JananiC_Intel]

Hi,

Thanks for detailed explanation.

For your information, if oneAPI AI toolkit is installed successfully the default tensorflow environment comes with that . In order to confirm the presence of tensorflow environment use the below command.

find -iname tensorflow

Usually it will be in "oneapi/2022.2/oneapi/intelpython/latest/envs/tensorflow" path.

Try the above command and share the results.

Regards,

Janani Chandran

[StephB]

Thanks for this. find yields a lot "permission denied" results as oneAPI is installed in opt, see below:

/opt/intel/oneapi/intelpython/latest/envs/tensorflow

/opt/intel/oneapi/intelpython/latest/envs/tensorflow-2.8.0

And I have added the following now: I use Spyder as my editor so there is a Spyder environment, see below. The ~\.coda\environments.txt file belonged to root so I have chown to steph.

/home/steph/anaconda3/envs/spyder-env

Trust this will help.

[JananiC_Intel]

Hi,

Thanks for the update.

Kindly follow the below steps. Also, the below steps will not cause any permission denied issues.

1)Go to tensorflow directory

cd /opt/intel/oneapi/intelpython/latest/envs/tensorflow

2)Activate tensorflow environment with the below command.

source vars.sh

After this 2nd step, your environment will get activated and you can proceed with the cloning steps which was mentioned earlier in this thread.

Regards,

Janani Chandran

[StephB]

There is no vars.sh file in this directory? What I do see is the following in the env directory:

steph@xxxxx:$ ls -l env/
total 24
-rwxrwxr-x 5 root root 10318 Mar 29 13:09 compiler_rt_vars.sh
-rw-rw-r-- 5 root root 11929 Jan 12 19:31 vars.sh
steph@xxxxx:$

Must I execute this file?

[JananiC_Intel]

Hi,

Thanks for the update.

Did you search in "/opt/intel/oneapi/intelpython/latest/envs/tensorflow/env" path ? Can you confirm?

In one of your previous posts, you have mentioned that you have tensorflow in "/opt/intel/oneapi/intelpython/latest/envs/tensorflow" path. If so, just give "ls" command from env directory and share the result. Normally there will be compiler_rt_vars.sh and vars.sh in env directory of tensorflow and once we source vars.sh, tensorflow environment will get activated . Refer the attached screenshot for reference.

Regards,

Janani Chandran

[StephB · Accepted solution]

I have shown the ls in env in my previous with results as follows

steph@xxxxx:$ ls -l env/
total 24
-rwxrwxr-x 5 root root 10318 Mar 29 13:09 compiler_rt_vars.sh
-rw-rw-r-- 5 root root 11929 Jan 12 19:31 vars.sh
steph@xxxxx:$

I wanted confirmation please that that is the file I must execute. I think have now confirmed that that is what I must do, I'll continue. Thanks again.

I have doen everything, seems I am sorted! Now I think I must install Spyder again in this new environment.

Edit:

I have installed Spyder and everything seems in order.

Many thanks again

[JananiC_Intel]

Hi,

Glad to know that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Regards,

Janani Chandran