Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
679 Discussions

OpenVINO demo hangs on iris_xe_max nodes

Elvis_F_Intel
Employee
555 Views

I have an OpenVINO application that hangs on the iris_xe_max nodes. Looking into it, I also realized that the demos do hang as well. Please find below directions to reproduce:

 

Build the openvino demo:

source /opt/intel/openvino/bin/setupvars.sh

cd /opt/intel/openvino/deployment_tools/inference_engine/demos

./build_demos.sh

(This should build the demos to a subdirectory in your home directory: ~/omz_demos_build/intel64/Release)

 

Get pre-trained models:

cd ~/omz_demos_build/intel64/Release

 wget https://download.01.org/opencv/2020/openvinotoolkit/2020.4/open_model_zoo/models_bin/1/face-detectio...

wget https://download.01.org/opencv/2020/openvinotoolkit/2020.4/open_model_zoo/models_bin/1/face-detectio...

wget https://download.01.org/opencv/2020/openvinotoolkit/2020.4/open_model_zoo/models_bin/1/head-pose-est...

wget https://download.01.org/opencv/2020/openvinotoolkit/2020.4/open_model_zoo/models_bin/1/head-pose-est...

 

Get a sample video:

git clone https://github.com/intel-iot-devkit/sample-videos.git

 

Run demo on login node: (THIS RUNS FINE AND RETURNS IN ABOUT A MINUTE OR TWO)

 

./interactive_face_detection_demo -i sample-videos/head-pose-face-detection-female-and-male.mp4 -m face-detection-adas-0001.xml -m_hp head-pose-estimation-adas-0001.xml -no_show

 

Run demo on iris_xe_max_node: (THIS HANGS AND DOESN’T RETURN)

qsub -I -l nodes=1:iris_xe_max:ppn=2,walltime=08:0:0

cd ~/omz_demos_build/intel64/Release

source /opt/intel/openvino/bin/setupvars.sh

./interactive_face_detection_demo -i sample-videos/head-pose-face-detection-female-and-male.mp4 -m face-detection-adas-0001.xml -m_hp head-pose-estimation-adas-0001.xml -no_show

./interactive_face_detection_demo -i sample-videos/head-pose-face-detection-female-and-male.mp4 -m face-detection-adas-0001.xml -m_hp head-pose-estimation-adas-0001.xml -no_show -d GPU

 

Any assistance is appreciated,

Thanks,

Elvis.

0 Kudos
11 Replies
Iffa_Intel
Moderator
540 Views

Greetings,

 

May I know which Linux distribution are you using? (eg Ubuntu 18.04).

Are you able to run the verification script ./demo_squeezenet_download_convert_run.sh. This is important to ensure that  you had set up the toolkit correctly as in here: https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html

 

Besides, It is recommended for you to use the latest OpenVINO toolkit version which is 2021.2, and ensures that you had the updated plugin for your hardware.

I had some issues when inferencing in Raspbian with NCS2 previously, but this latest version fixes it.

 

You may also get the latest model here:

https://download.01.org/opencv/2021/openvinotoolkit/2021.2/

 

Plus, for running the interactive_face_detection_demo try this:

./interactive_face_detection_demo -i 0 -m intel/face-detection-adas-0001/FP16/face-detection-adas-0001.xml -m_hp intel/head-pose-estimation-adas-0001/FP16/head-pose-estimation-adas-0001.xml -d GPU -d_hp GPU

 

 

Sincerely,

Iffa

 

Elvis_F_Intel
Employee
533 Views

Iffa,

Thanks for your response.

I am running on DevCloud. I checked the Linux distribution and it is Ubuntu 20.04.

I also downloaded OpenVINO 2021.2 and used that instead of the default OpenVINO installation on DevCloud. There was a slight improvement as the models were now loaded onto the CPU. However, it still hangs trying to load the models to the GPU. Find below the output:

~/omz_demos_build/intel64/Release$ ./interactive_face_detection_demo -i sample-videos/head-pose-face-detection-female-and-male.mp4 -m face-detection-adas-0001.xml -m_hp head-pose-estimation-adas-0001.xml -no_show -d GPU -d_hp GPU
InferenceEngine: API version ......... 2.1
Build ........... 2021.2.0-1877-176bdf51370-releases/2021/2
[ INFO ] Parsing input parameters
[ INFO ] Loading device GPU

(It hangs at this point)

By the way, I have run this application successfully on my local server, and also on other DevCloud nodes. This issue only occurs on nodes with iris_xe_max GPUs.

 

Hope this further information is helpful. Any further assistance is appreciated.

Thanks,

Elvis.

Iffa_Intel
Moderator
517 Views



Fyi, this is the Update for DG1 (Iris® Xe MAX Graphics) GPU is supported starting with OV 2021.2 release - https://software.intel.com/content/www/us/en/develop/articles/openvino-relnotes.html#inpage-nav-1-2-...


You may try to check prerequisites (GPU driver) as per https://docs.openvinotoolkit.org/latest/openvino_docs_install_guides_installing_openvino_linux.html#...



Sincerely,

Iffa

 


Elvis_F_Intel
Employee
503 Views

Iffa,

Thanks for the response.

I dug into it a little more and it looks like the issue is coming from the Inference Engine trying to query the devices on the iris_xe_max systems. 

This is evident when building and running "hello_query_device" located at  <OpenVINO directory>/deployment_tools/inference_engine/samples/cpp.

It hangs when running GetAvailableDevices() to get the devices on the iris_xe_max system.

Can you confirm that as of now, one can successfully query a device through OpenVINO on an iris_xe_max DevCloud system? (Or by extension run a demo using an iris_xe_max GPU?)

Is it possible to get someone on the DevCloud side of things to ensure that the drivers etc needed to run OpenVINO are in place on the iris_xe_max nodes?

 

Thanks once more for your assistance.

Elvis. 

 

Iffa_Intel
Moderator
499 Views

Greetings,


Since the Iris® Xe MAX Graphics GPU is supported starting with OV 2021.2 release , the inferencing should work but you need to use this specific version of OpenVINO(2021.2): - https://software.intel.com/content/www/us/en/develop/articles/openvino-relnotes.html#inpage-nav-1-2-...


You may do the following on OV 2021.2: install GPU pre-requisites as per https://docs.openvinotoolkit.org/2021.2/openvino_docs_install_guides_installing_openvino_linux.html#... and ensure this is installed without any errors.


Then to run the verification script (demo squeezenet with -d GPU parameter).


If you still see any error for the verification script, then please help to create a new ticket in DevCloud for Edge forum - https://community.intel.com/t5/Intel-DevCloud-for-Edge/bd-p/devcloud-edge


Sincerely,

Iffa


Elvis_F_Intel
Employee
478 Views

Iffa,

Thanks for your response.

As you can see from the output below, I am using OpenVINO 2021.2:

~/omz_demos_build/intel64/Release$ ./interactive_face_detection_demo -i sample-videos/head-pose-face-detection-female-and-male.mp4 -m face-detection-adas-0001.xml -m_hp head-pose-estimation-adas-0001.xml -no_show -d GPU -d_hp GPU
InferenceEngine: API version ......... 2.1
Build ........... 2021.2.0-1877-176bdf51370-releases/2021/2
[ INFO ] Parsing input parameters
[ INFO ] Loading device GPU

I don't know if creating a ticket on DevCloud for Edge will be helpful because as far as I know, there are no Iris_xe_max GPUs on DevCloud for Edge. The application runs fine on DevCloud for Edge using the integrated GPUs. The hangs only happen on DevCloud for oneAPI with the dedicated Iris_xe_max nodes. Please let me know where to file a bug report on DevCloud for oneAPI.

On another note, one cannot run install_NEO_OCL_driver.sh in the link you sent as a regular user, root privileges are required.

https://docs.openvinotoolkit.org/2021.2/openvino_docs_install_guides_installing_openvino_linux.html#... 

That's the reason I asked that you get someone on the DevCloud side of things to make sure all drivers are working properly on the Iris_xe_max nodes.

Like I mentioned previously, the Inference Engine fails when trying to query devices using GetAvailableDevices(). If you can get someone to verify and fix this, that will be great.

Thanks,

Elvis.

RaeesaM_Intel
Moderator
336 Views

Hi,


We have informed the DevCloud Admin team regarding the driver issue. We will get back to you on the updates.


Raeesa


RaeesaM_Intel
Moderator
313 Views

Hi ,


The issue with drivers have been resolved by the team. Could you please check whether the demo is working fine now .


Raeesa


Elvis_F_Intel
Employee
307 Views

Hi Raeesa,

Yes, updating the drivers fixed the issues. The application runs fine now.

Thanks,

Elvis.

RaeesaM_Intel
Moderator
302 Views

Hi,


Thank you for the response. Glad that your issue got resolved . We are discontinuing monitoring this thread. Please raise a new one incase of any issues.


Regards,

Raeesa


RaeesaM_Intel
Moderator
302 Views

Hi,


Thank you for the response. Glad that your issue got resolved . We are discontinuing monitoring this thread. Please raise a new one incase of any issues.


Regards,

Raeesa


Reply