About Multiple Neural Compute Stick and HDDL Plugin with Yolov3

[Aktas__Utku]

Hi,

I have a couple of questions about correct way to use sticks and the HDDL devices.

I am using Qt and Openvino R3.1. Yolov3 example running on one stick without problem(~2 FPS). In my application there will be multiple cameras assign to different number of sticks(HDDL devices). For now I can assign one stick for one camera.

To be able to use two stick for each camera, I tried using MULTI plugin in Yolo(async) sample, gave "MULTI:MYRIAD.3.2-ma2480,MYRIAD.3.4-ma2480" to Inference engine's loadNetwork but it dramatically reduced performance to 1 FPS. I tried to use multiple inference requests, but IInferRequest::WaitMode::RESULT_READY was blocking it. I change it to STATUS_READY which should be non blocking; but it did not returned the result. I tried multi threading, created 2 object of the same class, assigned those objects to different threads, it was still 2 FPS.

When I tried to use Mustang-V100-MX4, it was using 4 Myriad VPU with %20 utility.(Checked with hddldaemon)

I checked camera settings, so this is not an issue with the camera neither network.

1- Is there any tool that I can observe Neural Compute Stick Usage?

2- I have found a python sample which uses multiprocessing. Is there any other way to use 2 stick(or HDDL devices) with yolo sample with better performance?  Should I use inference request queue?

3- How can i assign number of cores in HDDL per camera and make sure cores work with almost %100 utility?

4- Will there be a problem when I use multiple HDDL devices with multiple cameras like Openvino may assigning whole HDDL device to one camera since It already 4 core with %20 percent? (Similar problem like device not found)

Thank you in advance...

Utku Aktas

[JesusE_Intel · Accepted solution]

Hi Utku Aktas,

Thanks for reaching out. Are you trying to use a Vision Accelerator (Mustang-V100-MX4) along side with the Intel Neural Compute Stick 2? Let me look into the YOLOv3 with HDDL.

1- Is there any tool that I can observe Neural Compute Stick Usage?

There is no tool to measure the Intel Neural Compute Stick 2 utilization.

 2- I have found a python sample which uses multiprocessing. Is there any other way to use 2 stick(or HDDL devices) with yolo sample with better performance?  Should I use inference request queue?

Let me check and get back to you on this one.

3- How can i assign number of cores in HDDL per camera and make sure cores work with almost %100 utility? 

These guides may help with this, in particular the workload distribution guide: https://docs.openvinotoolkit.org/latest/_docs_install_guides_movidius_setup_guide.html

4- Will there be a problem when I use multiple HDDL devices with multiple cameras like Openvino may assigning whole HDDL device to one camera since It already 4 core with %20 percent?

This should not be the case.

Regards,

Jesus