About HDDL Plugin [1.Parallel Request, 2.Initialization Popping up]

[Catastrophe]

I have a few questions regarding the HDDL Plugin, 

1) Parallel Requests

I am currently using the HDDL (with 8 VPUs) as the device for my network.

Before, in order to process 8 images in parallel, I used following format as the device, MULTI:MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1),MYRIAD.X.X.XXXX(1).

But when I tried to do a stress test, using the MULTI plugin causes the program to stop at some point. So, I tried HDDL plugin and it was stable. 

My problem is that when I use the HDDL Plugin, the 8 images are being distributed to only MYRIADs. (For example: MYRIAD1->4 requests, MYRIAD2->4requests)

I want to maximize the use of the 8 requests in order to shorten the processing time. I want to know if there is a way wherein I can assign 1 request to each MYRIAD when using the HDDL Plugin?

2) Initialization Popping up

Is there a way to minimize the command prompt popping up upon the initialization of the HDDL? Or is there even a way to suppress the popping up of the command prompt? 

[Peh_Intel]

Hi Catastrope,

Greetings to you.

For Intel® Movidius™ VPU, the recommended inference requests for each Executable Network to achieve optimum performance is four. That’s the reason why the eight inference requests are distributed to the first two MYRIADs.

However, you can try to manually state the number of inference requests in the coding. There is a num_requests argument in load_network(). You may refer to this link:

https://docs.openvinotoolkit.org/2021.2/ie_python_api/classie__api_1_1IECore.html#ac9a2e043d14ccfa9c6bbf626cfd69fcc

I don’t think we can minimize the command prompt popping up upon the initialization of the HDDL.

Regards,

Peh

[Catastrophe]

Hi, thanks for the reply. But I still have some follow-up questions,

However, you can try to manually state the number of inference requests in the coding. There is a num_requests argument in load_network(). 

-> The link that you have provided is in Python. Is there also a way to manually state the number of inference requests in C++?

-> Also is there a reason why the HDDL is more stable when executing for a long period of time? 

I don’t think we can minimize the command prompt popping up upon the initialization of the HDDL.

-> I hope there would be a fix on this because it is covering the GUI upon deployment.

[Peh_Intel]

Hi Catastrope,

Greetings to you. For stating number of requests in C++, you may refer to benchmark_app, the nireq variable in the section 9 "Creating infer requests and filling input blobs". https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/samples/benchmark_app/main.cpp#L487

It is recommended to use HDDL plugin for HDDL cards, since MYRIAD plugin is mostly intended for the prototyping purpose.

I am sorry for your inconvenience that covering the GUI upon deployment. However, this is the nature behavior of HDDL plugin that when the hddldaemon starts, it launches the standalone Autoboot process.

Regards,

Peh

[Peh_Intel]

Hi Catastrope,

This thread will no longer be monitored since we have provided a suggestion. If you need any additional information from Intel, please submit a new question.

Regards,

Peh