Inference on HDDL Plugin

[IDPBY]

I encountered a issue of inferencing on HDDL plugin. The fps and latency of inferencing is not consistent while using the OpenVino inference engine "object_detection_demo.py". I wonder why, and is there any solution for it. The problem are attached below:

Note: The content wise of the inference engine is same despite the naming is different.

[Peh_Intel]

Hi IDPBY,

Thanks for reaching out to us.

I believe you have configured Intel® Vision Accelerator Design with Intel® Movidius™ VPUs on Linux correctly.

On my side, I validated that the FPS and latency of inferencing is consistent while running Object Detection Python Demo using HDDL plugin. I am using OpenVINO™ 2021.4.752.

Please try to run the demo with OpenVINO™ Public Pre-Trained Model, yolo-v4-tf again. Besides, please also try inferencing with Benchmark Python Tool with the custom model and OpenVINO™ Public Pre-Trained Model using HDDL plugin.

Regards,

Peh

[IDPBY]

Hi @Peh_Intel ,

Thanks for your kind suggestion.

I tried inferencing using the Benchmark Python Tool with custom model using HDDL and CPU plugins. I found out that the latency and fps of both devices are about the same. Is it normal or is it caused by the VPUs that I used is not operating optimally? How to I check if the VPUs is fully operating. By the way, I am using UP Vision Plus X ( 3xMyriad X). 

Inferencing on HDDLInferencing on CPU

Note: I enable my VPUs (3x  Myriad X) as shown in following commands: 

cd /opt/intel/openvino/deployment_tools/inference_engine/external/hddl
./install_IVAD_VPU_dependencies.sh
cd /opt/intel/openvino/deployment_tools/inference_engine/external/hddl/drivers
sudo ./setup.sh install

sudo nano /opt/intel/openvino_2021/inference_engine/external/hddl/config/hddl_autoboot.config

set
"total_device_num": 3

/opt/intel/openvino/inference_engine/external/hddl/config/hddl_service.config
"device_snapshot_mode": "base".

[Peh_Intel]

Hi IDPBY,

When you first initialize the HDDL, all the MYRIADs will be booted and shown as the picture below.

Besides, you can also modify the hddl_service.config file in <OV>\3rdparty\hddl\config\hddl_service.config and change "device_snapshot_mode": "base", to "device_snapshot_mode": "full" to view the util % of all MYRIADs when inferencing.

After changing a configuration file, restart the hddldaemon in <OV>\3rdparty\hddl\bin and run your application with -d HDDL from another window. 

The following YouTube video walks you through the process, however, it's for an older release and some parts may have changed.

https://www.youtube.com/watch?v=nyBvFF_z8Sw

Regards,

Peh

[IDPBY]

Hi @Peh_Intel ,

Thank you so much. I configured the HDDL with the steps that you pointed out. However, I encountered another issue when I was trying to run the benchmark app on HDDL. I tried running the following command: ~/inference_engine_cpp_samples_build/intel64/Release/benchmark_app -d HDDL -m ~/openvino_models/ir/intel/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -i /opt/intel/openvino_2021/deployment_tools/demo/car.png

, and there are several errors displayed as below:

 

I did follow the suggestions bolded in "blue" color, but it cant really solve it. Is there any way to solve the above issues?

Thank you.

[Peh_Intel]

Hi IDPBY,

Please try to upgrade OpenVINO™ to 2022.1.0. and configure the HDDL again.

${HDDL_INSTALL_DIR}/install_IVAD_VPU_dependencies.sh

cd ${HDDL_INSTALL_DIR}/drivers

sudo ./setup.sh install

Here is the output of inferencing with UP AI Core XP4 (4 MYRIADs):

For your information, since the OpenVINO™ 2022.1 release, the following development tools: Model Optimizer, Post-Training Optimization Tool, Model Downloader and other Open Model Zoo tools, Accuracy Checker, and Annotation Converter are not part of the installer. These tools are now only available on pypi.org.

Regards,

Peh

[IDPBY]

Hi @Peh_Intel ,

Thank you for your suggestion. I tried to install OpenVINO 2020.1, but I encountered an issue when I was trying to run the build_samples.sh script (https://docs.openvino.ai/latest/openvino_docs_OV_UG_Samples_Overview.html). 

How can I solve this issue? 

[Peh_Intel]

Hi IDPBY,

Please ensure that you have install the external software dependencies and dependencies for the GPU:

1.      Go to the install_dependencies directory:

cd <INSTALL_DIR>/install_dependencies/

2.      Run a script to download and install the external software dependencies:

sudo -E ./install_openvino_dependencies.sh

3.      Install the Intel® Graphics Compute Runtime for OpenCL™ driver components required to use the GPU plugin and write custom layers for Intel® Integrated Graphics. The drivers are not included in the package. To install, run this script:

sudo -E ./install_NEO_OCL_driver.sh

Regards,

Peh

[IDPBY]

Hi @Peh_Intel ,

I have installed all the software and hardware dependencies. I wish to use the HDDL hardware as inferencing tool. I tried to build the C sample application, it worked and built successfully. But for the C++ sample application, it showed the following issues, and I was not able to build it successfully. I would like to use the benchmark_app in the C++ sample application to evaluate the performance.

Is it a software bug which causes the command C++ build_sample.sh fail.

 

[Peh_Intel]

Hi IDPBY,

I have just uninstalled and re-installed OpenVINO™ 2022.1.0. I was able to build the samples on my side.

From your error, please have a check with the /opt/intel/openvino_2022/samples/cpp/thirdparty/json/nlohmann_json/single_include/ directory.

Please also have a try to uninstall and re-installed OpenVINO™ 2022.1.0. again.

Regards,

Peh

[IDPBY]

Hi @Peh_Intel ,

I got an error which the directory of the install TARGETS given target "nlohmann" file does not exist. The nlohmann file actually exists in that given directory. I wonder why? 

By the way, I am using Ubuntu 18.04 version and I have both 2021.4 and 2022.1 OpenVINO version. If you don't mind, could you guide me step by step starting from installation until completion of benchmark_app setup. I am worried that I missed out several steps which caused the above error.

Thank you.

[Peh_Intel]

Hi IDPBY,

I also encountered the same error when building the application in Ubuntu 18.04.

It seems like the error is due to the default CMake version (3.10.2).

Please uninstall the CMake and then install the latest CMake version.

1. sudo apt remove cmake

2. wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add –

3. sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic main'

4. sudo apt-get update

5. sudo apt install cmake

Regards,

Peh

[IDPBY]

Dear @Peh_Intel ,

I used the benchmark app in OpenVINO 2022.1 version to do inferencing on HDDL plugin. However, the errors are still same as shown in OpenVINO 2021.4 Version. Is it possible that the hardware not compatible with the software?

PS: The above three pictures are the errors shown in OpenVINO 2021.4 Version. 

[Peh_Intel · Accepted solution]

Hi IDPBY,

The incompatible of hardware with software is not the case as we are configuring the same steps for Intel® Vision Accelerator Design with Intel® Movidius™ VPUs.

Is there any chance for you to try with other HDDL cards to eliminate the possibility of the hardware malfunction issue?

Regards,

Peh

[Peh_Intel]

Hi IDPBY,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored. 

Regards,

Peh