Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1644 Discussions

Unable to import pandas after activating openvino_2022.2.0_python3.6_RC2_TF1.15

Wen_Jie
Beginner
1,017 Views

Greeting to everyone

 

I was doing a job submission to an edge node with an Intel® Xeon® Gold 6258R Processor by writing a .sh file and using qsub to submit.

Wen_Jie_0-1682229692785.png

Wen_Jie_1-1682229711153.png

 

After that, the message is come out as shown below.

Wen_Jie_2-1682229811316.png

Wen_Jie_3-1682229903797.png

 

After the OpenVino environment is initialized, the python script started to run.

ModuleNotFoundError: No module named 'pandas'  appear.

I even try to install pandas into the virtual environment by using the terminal.

Wen_Jie_4-1682230052402.png

However, the same error is still there.

What should I do to fix this?

 

 

Yours Sincerely,

Wen Jie

 

 

Labels (1)
0 Kudos
1 Solution
JesusE_Intel
Moderator
893 Views

Hi Wen Jie,

 

I see you are trying to install the package on the pre-configured kernels which you may not have permissions to make changes. I suggest creating your own custom kernel and install the packages you need. Please follow these steps to create a custom kernel, add it to JupyterLab and submit the job to an edge node.

 

1. Open the Terminal on Intel Developer Cloud for the Edge

2. Create a virtual environment in your home directory and install the required packages

virtualenv --python=/usr/bin/python3.8 ~/venv/openvino_2022.2.0_python3.8_custom
source ~/venv/openvino_2022.2.0_python3.8_custom/bin/activate
pip install -U pip
pip install wheel jupyterhub jupyterlab-widgets==1.1.0 widgetsnbextension==3.6.0
pip install openvino==2022.2.0 tensorflow pandas seaborn

3. Create the kernel file

mkdir -p ~/.local/share/jupyter/kernels/custom
cat > ~/.local/share/jupyter/kernels/custom/kernel.json <<EOF
{
        "argv":[
                "$HOME/venv/openvino_2022.2.0_python3.8_custom/bin/python3",
                "-m",
                "ipykernel_launcher",
                "-f",
                "{connection_file}"
        ],
        "display_name": "Python3.8 (OpenVINO 2022.2.0) Custom",
        "language": "python",
        "env": {"PBS_DEFAULT": "v-qsvr-1", "VENV_PATH": "$HOME/venv/openvino_2022.2.0_python3.8_custom"}
}
EOF

4. From the top menu, click File -> Hub Control Panel and stop the Server.

5. Click Start My Server

6. Create a new Jupyter Notebook and select the custom kernel from the top right dropdown.

7. Copy the following code into individual cells

%%writefile main.py
import pandas
import seaborn

print("Done Importing")
%%writefile run.sh
cd $PBS_O_WORKDIR
source $HOME/venv/openvino_2022.2.0_python3.8_custom/bin/activate
python3 --version
python3 main.py
#Submit job to the queue
job_id_cpu = !qsub run.sh -l nodes=1:idc082 -N test 
print(job_id_cpu[0])

8. Open the job output file ( e.g. custom_kernel_job.o851554) and confirm you see the following

# Resources: neednodes=1:idc082,nodes=1:idc082,walltime=01:00:00
########################################################################

Python 3.8.0
Done Importing

########################################################################
# End of output for job 851554.v-qsvr-1.devcloud-edge

 

Regards,

Jesus

View solution in original post

0 Kudos
4 Replies
JesusE_Intel
Moderator
948 Views

Hi Wen Jie,


I was able to reproduce the issue when submitting a job to a couple of nodes. Could you try to submit the job to IDC-052/ Intel® Xeon® Gold 6338N (Ice Lake) and see if you run into the module error? I did not see the issue on this node, I'll need to check with the development team on this.


In the meantime, let me know if that node works for you.


Regards,

Jesus


0 Kudos
Wen_Jie
Beginner
932 Views

Hi Jesus

 

I have tried to submit to IDC-052/ Intel® Xeon® Gold 6338N (Ice Lake), and it is working, the ModuleNotFoundError: No module named 'pandas' is solved.

 

However, another module error was found. 

ModuleNotFoundError: No module named 'seaborn'

Wen_Jie_0-1682480923262.png

I also tried to install using the terminal.

Wen_Jie_1-1682481363952.png

 

How can I solve this?

 

 

Regards

Wen Jie

0 Kudos
JesusE_Intel
Moderator
894 Views

Hi Wen Jie,

 

I see you are trying to install the package on the pre-configured kernels which you may not have permissions to make changes. I suggest creating your own custom kernel and install the packages you need. Please follow these steps to create a custom kernel, add it to JupyterLab and submit the job to an edge node.

 

1. Open the Terminal on Intel Developer Cloud for the Edge

2. Create a virtual environment in your home directory and install the required packages

virtualenv --python=/usr/bin/python3.8 ~/venv/openvino_2022.2.0_python3.8_custom
source ~/venv/openvino_2022.2.0_python3.8_custom/bin/activate
pip install -U pip
pip install wheel jupyterhub jupyterlab-widgets==1.1.0 widgetsnbextension==3.6.0
pip install openvino==2022.2.0 tensorflow pandas seaborn

3. Create the kernel file

mkdir -p ~/.local/share/jupyter/kernels/custom
cat > ~/.local/share/jupyter/kernels/custom/kernel.json <<EOF
{
        "argv":[
                "$HOME/venv/openvino_2022.2.0_python3.8_custom/bin/python3",
                "-m",
                "ipykernel_launcher",
                "-f",
                "{connection_file}"
        ],
        "display_name": "Python3.8 (OpenVINO 2022.2.0) Custom",
        "language": "python",
        "env": {"PBS_DEFAULT": "v-qsvr-1", "VENV_PATH": "$HOME/venv/openvino_2022.2.0_python3.8_custom"}
}
EOF

4. From the top menu, click File -> Hub Control Panel and stop the Server.

5. Click Start My Server

6. Create a new Jupyter Notebook and select the custom kernel from the top right dropdown.

7. Copy the following code into individual cells

%%writefile main.py
import pandas
import seaborn

print("Done Importing")
%%writefile run.sh
cd $PBS_O_WORKDIR
source $HOME/venv/openvino_2022.2.0_python3.8_custom/bin/activate
python3 --version
python3 main.py
#Submit job to the queue
job_id_cpu = !qsub run.sh -l nodes=1:idc082 -N test 
print(job_id_cpu[0])

8. Open the job output file ( e.g. custom_kernel_job.o851554) and confirm you see the following

# Resources: neednodes=1:idc082,nodes=1:idc082,walltime=01:00:00
########################################################################

Python 3.8.0
Done Importing

########################################################################
# End of output for job 851554.v-qsvr-1.devcloud-edge

 

Regards,

Jesus

0 Kudos
JesusE_Intel
Moderator
768 Views

If you need any additional information, please submit a new question as this thread will no longer be monitored.


0 Kudos
Reply