Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Inference on HDDL Plugin

IDPBY
Beginner
1,379 Views

I encountered a issue of inferencing on HDDL plugin. The fps and latency of inferencing is not consistent while using the OpenVino inference engine "object_detection_demo.py". I wonder why, and is there any solution for it. The problem are attached below:

Openvino issue.jpg

Note: The content wise of the inference engine is same despite the naming is different.

0 Kudos
1 Solution
Peh_Intel
Moderator
1,136 Views

Hi IDPBY,


The incompatible of hardware with software is not the case as we are configuring the same steps for Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.


Is there any chance for you to try with other HDDL cards to eliminate the possibility of the hardware malfunction issue?



Regards,

Peh


View solution in original post

0 Kudos
14 Replies
Peh_Intel
Moderator
1,343 Views

Hi IDPBY,


Thanks for reaching out to us.


I believe you have configured Intel® Vision Accelerator Design with Intel® Movidius™ VPUs on Linux correctly.


On my side, I validated that the FPS and latency of inferencing is consistent while running Object Detection Python Demo using HDDL plugin. I am using OpenVINO™ 2021.4.752.


Please try to run the demo with OpenVINO™ Public Pre-Trained Model, yolo-v4-tf again. Besides, please also try inferencing with Benchmark Python Tool with the custom model and OpenVINO™ Public Pre-Trained Model using HDDL plugin.



Regards,

Peh


0 Kudos
IDPBY
Beginner
1,336 Views

Hi @Peh_Intel ,

 

Thanks for your kind suggestion.

 

I tried inferencing using the Benchmark Python Tool with custom model using HDDL and CPU plugins. I found out that the latency and fps of both devices are about the same. Is it normal or is it caused by the VPUs that I used is not operating optimally? How to I check if the VPUs is fully operating. By the way, I am using UP Vision Plus X ( 3xMyriad X). 

Inferencing on HDDLInferencing on HDDLInferencing on CPUInferencing on CPU

Note: I enable my VPUs (3x  Myriad X) as shown in following commands: 

cd /opt/intel/openvino/deployment_tools/inference_engine/external/hddl
./install_IVAD_VPU_dependencies.sh
cd /opt/intel/openvino/deployment_tools/inference_engine/external/hddl/drivers
sudo ./setup.sh install

sudo nano /opt/intel/openvino_2021/inference_engine/external/hddl/config/hddl_autoboot.config

set
"total_device_num": 3

/opt/intel/openvino/inference_engine/external/hddl/config/hddl_service.config
"device_snapshot_mode": "base".

0 Kudos
Peh_Intel
Moderator
1,320 Views

Hi IDPBY,

 

When you first initialize the HDDL, all the MYRIADs will be booted and shown as the picture below.

2_MYRIAD.JPG

 

Besides, you can also modify the hddl_service.config file in <OV>\3rdparty\hddl\config\hddl_service.config and change "device_snapshot_mode": "base", to "device_snapshot_mode": "full" to view the util % of all MYRIADs when inferencing.

 

After changing a configuration file, restart the hddldaemon in <OV>\3rdparty\hddl\bin and run your application with -d HDDL from another window. 

 

The following YouTube video walks you through the process, however, it's for an older release and some parts may have changed.

https://www.youtube.com/watch?v=nyBvFF_z8Sw

 

 

Regards,

Peh

 

0 Kudos
IDPBY
Beginner
1,314 Views

Hi @Peh_Intel ,

 

Thank you so much. I configured the HDDL with the steps that you pointed out. However, I encountered another issue when I was trying to run the benchmark app on HDDL. I tried running the following command: ~/inference_engine_cpp_samples_build/intel64/Release/benchmark_app -d HDDL -m ~/openvino_models/ir/intel/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -i /opt/intel/openvino_2021/deployment_tools/demo/car.png

, and there are several errors displayed as below:

WhatsApp Image 2022-06-07 at 4.34.21 PM.jpegWhatsApp Image 2022-06-07 at 4.28.42 PM.jpegWhatsApp Image 2022-06-07 at 4.28.21 PM.jpeg 

I did follow the suggestions bolded in "blue" color, but it cant really solve it. Is there any way to solve the above issues?

 

Thank you.

 

0 Kudos
Peh_Intel
Moderator
1,281 Views

Hi IDPBY,

 

Please try to upgrade OpenVINO™ to 2022.1.0. and configure the HDDL again.

${HDDL_INSTALL_DIR}/install_IVAD_VPU_dependencies.sh

cd ${HDDL_INSTALL_DIR}/drivers

sudo ./setup.sh install

 

 

Here is the output of inferencing with UP AI Core XP4 (4 MYRIADs):

inference.png

 

For your information, since the OpenVINO™ 2022.1 release, the following development tools: Model Optimizer, Post-Training Optimization Tool, Model Downloader and other Open Model Zoo tools, Accuracy Checker, and Annotation Converter are not part of the installer. These tools are now only available on pypi.org.

 

 

Regards,

Peh

 

0 Kudos
IDPBY
Beginner
1,252 Views

Hi @Peh_Intel ,

 

Thank you for your suggestion. I tried to install OpenVINO 2020.1, but I encountered an issue when I was trying to run the build_samples.sh script (https://docs.openvino.ai/latest/openvino_docs_OV_UG_Samples_Overview.html). 

 

bug.jpg

How can I solve this issue? 

0 Kudos
Peh_Intel
Moderator
1,218 Views

Hi IDPBY,

 

Please ensure that you have install the external software dependencies and dependencies for the GPU:

1.      Go to the install_dependencies directory:

cd <INSTALL_DIR>/install_dependencies/

 

 

2.      Run a script to download and install the external software dependencies:

sudo -E ./install_openvino_dependencies.sh

 

3.      Install the Intel® Graphics Compute Runtime for OpenCL™ driver components required to use the GPU plugin and write custom layers for Intel® Integrated Graphics. The drivers are not included in the package. To install, run this script:

sudo -E ./install_NEO_OCL_driver.sh

 

 

Regards,

Peh

 

0 Kudos
IDPBY
Beginner
1,209 Views

Hi @Peh_Intel ,

 

I have installed all the software and hardware dependencies. I wish to use the HDDL hardware as inferencing tool. I tried to build the C sample application, it worked and built successfully. But for the C++ sample application, it showed the following issues, and I was not able to build it successfully. I would like to use the benchmark_app in the C++ sample application to evaluate the performance.

IDPBY_0-1655103506111.png42f2d070-82a3-4b7c-9007-2fe63e791779.jpg

Is it a software bug which causes the command C++ build_sample.sh fail.

 

0 Kudos
Peh_Intel
Moderator
1,202 Views

Hi IDPBY,

 

I have just uninstalled and re-installed OpenVINO™ 2022.1.0. I was able to build the samples on my side.

Build.JPG

 

From your error, please have a check with the /opt/intel/openvino_2022/samples/cpp/thirdparty/json/nlohmann_json/single_include/ directory.

 

Please also have a try to uninstall and re-installed OpenVINO™ 2022.1.0. again.

 

 

Regards,

Peh

 

0 Kudos
IDPBY
Beginner
1,194 Views

Hi @Peh_Intel ,

 

I got an error which the directory of the install TARGETS given target "nlohmann" file does not exist. The nlohmann file actually exists in that given directory. I wonder why? 

 

By the way, I am using Ubuntu 18.04 version and I have both 2021.4 and 2022.1 OpenVINO version. If you don't mind, could you guide me step by step starting from installation until completion of benchmark_app setup. I am worried that I missed out several steps which caused the above error.

 

Thank you.

0 Kudos
Peh_Intel
Moderator
1,179 Views

Hi IDPBY,


I also encountered the same error when building the application in Ubuntu 18.04.


It seems like the error is due to the default CMake version (3.10.2).


Please uninstall the CMake and then install the latest CMake version.

1. sudo apt remove cmake

2. wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add –

3. sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

4. sudo apt-get update

5. sudo apt install cmake



Regards,

Peh


0 Kudos
IDPBY
Beginner
1,153 Views

Dear @Peh_Intel ,

 

I used the benchmark app in OpenVINO 2022.1 version to do inferencing on HDDL plugin. However, the errors are still same as shown in OpenVINO 2021.4 Version. Is it possible that the hardware not compatible with the software?

IDPBY_0-1655305608151.jpeg

IDPBY_1-1655305608365.jpg

IDPBY_2-1655305608260.jpeg

PS: The above three pictures are the errors shown in OpenVINO 2021.4 Version. 

 

 

0 Kudos
Peh_Intel
Moderator
1,137 Views

Hi IDPBY,


The incompatible of hardware with software is not the case as we are configuring the same steps for Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.


Is there any chance for you to try with other HDDL cards to eliminate the possibility of the hardware malfunction issue?



Regards,

Peh


0 Kudos
Peh_Intel
Moderator
1,084 Views

Hi IDPBY,


Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored. 



Regards,

Peh


0 Kudos
Reply