How to verify HDDL successful run on XM 2280 (OpenVINO 2020.4 with Ubuntu 18.04)

[thamml]

I need an assistance on verifying HDDL run on XM2280.

My OS is Linux version 5.4.0-48-generic (buildd@lcy01-amd64-023) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04))

I have completed VPU installation by following the link below
https://docs.openvinotoolkit.org/latest/_docs_install_guides_installing_openvino_linux_ivad_vpu.html 

I tested the command ./demo_security_barrier_camera.sh -d HDDL

The output shows 0 FPS as shown below ( note that I also ran with GPU, CPU and MYRIAD; there are  some values for FPS)

After I closed it, it shows some error message as below. Are the "bold" error message normal?

Run Inference Engine security_barrier_camera demo

Run ./security_barrier_camera_demo -d HDDL -d_va HDDL -d_lpr HDDL -i /opt/intel/openvino/deployment_tools/demo/car_1.bmp -m /home/thamml/openvino_models/ir/intel/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -m_lpr /home/thamml/openvino_models/ir/intel/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml -m_va /home/thamml/openvino_models/ir/intel/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml

[ INFO ] InferenceEngine: 0x7f818cd57030
[ INFO ] Files were added: 1
[ INFO ] /opt/intel/openvino/deployment_tools/demo/car_1.bmp
[ INFO ] Loading device HDDL
HDDL
HDDLPlugin version ......... 2.1
Build ........... 2020.4.0-359-21e092122f4-releases/2020/4

[ INFO ] Loading detection model to the HDDL plugin
[22:43:50.5576][2993]I[ClientManager.cpp:159] client(id:9) registered: clientName=HDDLPlugin socket=2
[22:43:50.9051][2994]I[GraphManager.cpp:491] Load graph success, graphId=9 graphName=vehicle-license-plate-detection-barrier-0106
[ INFO ] Loading Vehicle Attribs model to the HDDL plugin
[22:43:50.9474][2993]I[ClientManager.cpp:159] client(id:10) registered: clientName=HDDLPlugin socket=3
[22:43:50.9953][2994]I[GraphManager.cpp:491] Load graph success, graphId=10 graphName=vehicle-attributes-recognition-barrier-0039
[ INFO ] Loading Licence Plate Recognition (LPR) model to the HDDL plugin
[22:43:51.0078][2993]I[ClientManager.cpp:159] client(id:11) registered: clientName=HDDLPlugin socket=4
[22:43:51.0849][2994]I[GraphManager.cpp:491] Load graph success, graphId=11 graphName=LPRNet
[ INFO ] Number of InferRequests: 1 (detection), 3 (classification), 3 (recognition)
[ INFO ] Display resolution: 1920x1080
[ INFO ] Number of allocated frames: 3
[ INFO ] Resizable input with support of ROI crop and auto resize is disabled
^Cerror at /home/jenkins/agent/workspace/IE-Packages/BuildAndPush/hddl-service/utils/platform/linux/IPCLinux.cpp:262
read socket(idx=4) failed, user=HDDLPlugin leftBytes=4
(2) No such file or directory
[22:46:49.1413][3004]ERROR[hddl_protocol.cpp:153] read socket(idx=4) failed, user=HDDLPlugin leftBytes=4
[22:46:49.1415][3004]I[MessageDispatcher.cpp:209] failed to receive hddl message from socket (4)
error at /home/jenkins/agent/workspace/IE-Packages/BuildAndPush/hddl-service/utils/platform/linux/IPCLinux.cpp:262
read socket(idx=3) failed, user=HDDLPlugin leftBytes=4
(2) No such file or directory
[22:46:49.1421][3004]ERROR[hddl_protocol.cpp:153] read socket(idx=3) failed, user=HDDLPlugin leftBytes=4
[22:46:49.1422][3004]I[MessageDispatcher.cpp:209] failed to receive hddl message from socket (3)
error at /home/jenkins/agent/workspace/IE-Packages/BuildAndPush/hddl-service/utils/platform/linux/IPCLinux.cpp:262
read socket(idx=2) failed, user=HDDLPlugin leftBytes=4
(2) No such file or directory
[22:46:49.1428][3004]ERROR[hddl_protocol.cpp:153] read socket(idx=2) failed, user=HDDLPlugin leftBytes=4
[22:46:49.1429][3004]I[MessageDispatcher.cpp:209] failed to receive hddl message from socket (2)

I also tested 2nd command ./classification_sample_async -i /opt/intel/openvino/deployment_tools/demo/car.png -m ~/openvino_models/ir/public/squeezenet1.1/FP16/squeezenet1.1.xml -d HDDL

below is the output

[ INFO ] InferenceEngine:
API version ............ 2.1
Build .................. 2020.4.0-359-21e092122f4-releases/2020/4
Description ....... API
[ INFO ] Parsing input parameters
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ] /opt/intel/openvino/deployment_tools/demo/car.png
[ INFO ] Creating Inference Engine
HDDL
HDDLPlugin version ......... 2.1
Build ........... 2020.4.0-359-21e092122f4-releases/2020/4

[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ WARNING ] Image is resized from (787, 259) to (227, 227)
[ INFO ] Batch size is 1
[ INFO ] Loading model to the device
[22:28:46.7940][2993]I[ClientManager.cpp:159] client(id:8) registered: clientName=HDDLPlugin socket=2
[22:28:46.8827][2994]I[GraphManager.cpp:491] Load graph success, graphId=8 graphName=squeezenet1.1
[ INFO ] Create infer request
[ INFO ] Start inference (10 asynchronous executions)
[ INFO ] Completed 1 async request execution
[ INFO ] Completed 2 async request execution
[ INFO ] Completed 3 async request execution
[ INFO ] Completed 4 async request execution
[ INFO ] Completed 5 async request execution
[ INFO ] Completed 6 async request execution
[ INFO ] Completed 7 async request execution
[ INFO ] Completed 8 async request execution
[ INFO ] Completed 9 async request execution
[ INFO ] Completed 10 async request execution
[ INFO ] Processing output blobs

Top 10 results:

Image /opt/intel/openvino/deployment_tools/demo/car.png

classid probability label
------- ----------- -----
817 0.6708984 sports car, sport car
479 0.1922607 car wheel
511 0.0936890 convertible
436 0.0216064 beach wagon, station wagon, wagon, estate car, beach waggon, station waggon, waggon
751 0.0075760 racer, race car, racing car
656 0.0049667 minivan
717 0.0027428 pickup, pickup truck
581 0.0019779 grille, radiator grille
468 0.0014219 cab, hack, taxi, taxicab
661 0.0008636 Model T

[22:28:46.9894][2993]I[ClientManager.cpp:189] client(id:8) unregistered: clientName=HDDLPlugin socket=2
[22:28:46.9958][2994]I[GraphManager.cpp:539] graph(8) destroyed
[ INFO ] Execution successful

[ INFO ] This sample is an API example, for any performance measurements please use the dedicated benchmark_app tool

Kindly advise. 

Tham

[Rizal_Intel]

Hi Tham,

I will try to replicate and investigate your issue.

I will get back to you soon.

Regards,

Rizal

[Rizal_Intel]

Hi Tham,

I ran the demo using HDDL plugin, I did encounter any error but I get the same result of 0 FPS.

For HDDL plugin the demo requires some further configuration.

I am using a different HDDL device (Mustang V100) but it should not be an issue.

Regards,

Rizal

[thamml]

Hi,

I now cant even load the HDDL device. When I run the command

./security_barrier_camera_demo -d HDDL -d_va HDDL -d_lpr HDDL -n_iqs 10 -n_wt 4 -nireq 10 -i /opt/intel/openvino/deployment_tools/demo/car_1.bmp -m ~/openvino_models/ir/intel/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -m_lpr ~/openvino_models/ir/intel/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml -m_va ~/openvino_models/ir/intel/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml
[ INFO ] InferenceEngine: 0x7f97dadaa030
[ INFO ] Files were added: 1
[ INFO ] /opt/intel/openvino/deployment_tools/demo/car_1.bmp
[ INFO ] Loading device HDDL
HDDL
HDDLPlugin version ......... 2.1
Build ........... 2020.4.0-359-21e092122f4-releases/2020/4

[ INFO ] Loading detection model to the HDDL plugin
[17:58:45.2120][1952]I[main.cpp:246] ## HDDL_INSTALL_DIR: /opt/intel/openvino_2020.4.287/deployment_tools/inference_engine/external/hddl
[17:58:45.2122][1952]I[main.cpp:248] Config file '/opt/intel/openvino_2020.4.287/deployment_tools/inference_engine/external/hddl/config/hddl_service.config' has been loaded
[17:58:45.2131][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_service_alive.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2132][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_service_ready.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2133][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_service_failed.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2134][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_start_exit.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2154][1952]I[AutobootStarter.cpp:156] Info: No running autoboot process. Start autoboot daemon...
[17:58:45.2354][1954]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_autoboot_alive.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2355][1954]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_autoboot_ready.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2356][1954]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_autoboot_start_exit.mutex owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2357][1954]I[FileHelper.cpp:272] Set file:/tmp/hddl_autoboot_device.map owner: user-'no_change', group-'users', mode-'0660'
[17:58:45.2359][1954]I[AutoBoot.cpp:308] [Firmware Config] deviceName=default deviceNum=0 firmwarePath=/opt/intel/openvino_2020.4.287/deployment_tools/inference_engine/external/hddl/lib/mvnc/usb-ma2x8x.mvcmd
[17:58:46.5873][1963]I[AutoBoot.cpp:197] Start boot device 3.1-ma2480
[17:58:46.8253][1963]I[AutoBoot.cpp:199] Device 3.1-ma2480 boot success, firmware=/opt/intel/openvino_2020.4.287/deployment_tools/inference_engine/external/hddl/lib/mvnc/usb-ma2x8x.mvcmd
[17:58:47.8257][1952]I[AutobootStarter.cpp:85] Info: Autoboot is running.
[17:58:47.8306][1952]W[ConfigParser.cpp:269] Warning: Cannot find key, path=scheduler_config.max_graph_per_device subclass=0, use default value: 1.
[17:58:47.8307][1952]W[ConfigParser.cpp:292] Warning: Cannot find key, path=scheduler_config.use_sgad_by_default subclass=0, use default value: false.
[17:58:47.8307][1952]I[DeviceSchedulerFactory.cpp:56] Info: ## DeviceSchedulerFacotry ## Created Squeeze Device-Scheduler2.
[17:58:47.8321][1952]I[DeviceManager.cpp:551] ## SqueezeScheduler created ##
[17:58:47.8321][1952]I[DeviceManager.cpp:649] times 0: try to create worker on device(4.1)
_name_mapping open failed err=2,No such file or directory
E: [ncAPI] [ 0] ncDeviceOpenBooted:892 Failed to connect device

[17:58:48.8338][1952]ERROR[HddlMyriadXDevice.cpp:129] Error: ncDeviceOpenBooted() failed, device=4.1 rc=-2
[17:58:48.8339][1952]ERROR[WorkThread.cpp:361] Error: open device failed, rc=-2
[17:58:48.8340][1952]ERROR[DeviceManager.cpp:652] [FAILED] times 0: create worker on device(4.1)
[17:58:49.8348][1952]I[DeviceManager.cpp:649] times 1: try to create worker on device(4.1)
_name_mapping open failed err=2,No such file or directory
E: [ncAPI] [ 0] ncDeviceOpenBooted:892 Failed to connect device

[17:58:50.8356][1952]ERROR[HddlMyriadXDevice.cpp:129] Error: ncDeviceOpenBooted() failed, device=4.1 rc=-2
[17:58:50.8358][1952]ERROR[WorkThread.cpp:361] Error: open device failed, rc=-2
[17:58:50.8358][1952]ERROR[DeviceManager.cpp:652] [FAILED] times 1: create worker on device(4.1)
[17:58:50.8361][1952]ERROR[DeviceManager.cpp:663] Don't find device 4.1, please restart service!!!
[17:58:50.8362][1952]I[DeviceManager.cpp:145] DEVICE FOUND : 0
[17:58:50.8362][1952]I[DeviceManager.cpp:146] DEVICE OPENED : 0
[17:58:50.8364][1952]I[DeviceManagerCreator.cpp:81] New device manager(DeviceManager0) created with subclass(0), deviceCount(1)
[17:58:50.8393][1952]I[TaskSchedulerFactory.cpp:45] Info: ## TaskSchedulerFactory ## Created Polling Task-Scheduler.
[17:58:50.8406][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_snapshot.sock owner: user-'no_change', group-'users', mode-'0660'
[17:58:50.8415][1952]I[FileHelper.cpp:272] Set file:/var/tmp/hddl_service.sock owner: user-'no_change', group-'users', mode-'0660'
[17:58:50.8418][1952]I[MessageDispatcher.cpp:87] Message Dispatcher initialization finished
[17:58:50.8419][1952]I[main.cpp:106] SERVICE IS READY ...
[17:58:50.8646][1990]I[ClientManager.cpp:159] client(id:1) registered: clientName=HDDLPlugin socket=2
[17:58:50.8687][1990]I[ClientManager.cpp:189] client(id:1) unregistered: clientName=HDDLPlugin socket=2
[ ERROR ] no devices found

Kindly advise.

[thamml]

Hi Rizal,

I can load the HDDL by reinstalling it (after referring to other posts). I ran the following command

./security_barrier_camera_demo -d HDDL -d_va HDDL -d_lpr HDDL -n_iqs 10 -n_wt 4 -nireq 10 -i /opt/intel/openvino/deployment_tools/demo/car_1.bmp -m ~/openvino_models/ir/intel/vehicle-license-plate-detection-barrier-0106/FP16/vehicle-license-plate-detection-barrier-0106.xml -m_lpr ~/openvino_models/ir/intel/license-plate-recognition-barrier-0001/FP16/license-plate-recognition-barrier-0001.xml -m_va ~/openvino_models/ir/intel/vehicle-attributes-recognition-barrier-0039/FP16/vehicle-attributes-recognition-barrier-0039.xml

The output is

[ INFO ] InferenceEngine: 0x7fef75f84030
[ INFO ] Files were added: 1
[ INFO ] /opt/intel/openvino/deployment_tools/demo/car_1.bmp
[ INFO ] Loading device HDDL
HDDL
HDDLPlugin version ......... 2.1
Build ........... 2020.4.0-359-21e092122f4-releases/2020/4

[ INFO ] Loading detection model to the HDDL plugin
[ INFO ] Loading Vehicle Attribs model to the HDDL plugin
[ INFO ] Loading Licence Plate Recognition (LPR) model to the HDDL plugin
[ INFO ] Number of InferRequests: 10 (detection), 30 (classification), 30 (recognition)
[ INFO ] Display resolution: 1920x1080
[ INFO ] Number of allocated frames: 10
[ INFO ] Resizable input with support of ROI crop and auto resize is disabled

without any error. But it gives 0.1 FPS as below.

My questions are:

1. You mentioned you had obtained the same result of 0 FPS (is this normal)?

2. How about my 0.1 FPS here?

3. Any other method to "appraise" the inference speed of VPU? What should be its expected performance?

According to https://docs.openvinotoolkit.org/2020.4/openvino_docs_IE_DG_supported_plugins_HDDL.html

It provides dozens amount of throughput as MYRIAD Plugin. How can I verify this?

[Rizal_Intel · Accepted solution]

Hi Tham,

The 0 FPS is expected with this script as it is only running inference on 1 frame and HDDL requires additional settings. Please see the readme file for this demo: https://github.com/openvinotoolkit/open_model_zoo/blob/master/demos/security_barrier_camera_demo/README.md#running regarding HDDL.

The best way to measure the performance of the plugin is to use the benchmark app.

You can find this in <openvino directory>/deployment_tools/demo/demo_benchmark_app.sh

This should help you compare the performance of different models and devices.

Regards,

Rizal

[Rizal_Intel]

Hi Tham,

Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Rizal