Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Hyodo__Katsuya
Innovator
1,560 Views

How to use Multi Stick NCS2 (Parallelization of reasoning)

Jump to solution

Hello. Expert engineers.

In the past I have created a multi-stick + multi-process high performance Python program using NCSDK2.
https://github.com/PINTO0309/MobileNet-SSD-RealSense.git

However, now I am working on migration from NCSDK2 to OpenVINO.
My environment is as follows.

- OpenVINO R5 2018.5.445
- RaspberryPi3 (Raspbian Stretch)
- OpenCV 4.0.1-openvino
- Python3.5
- Neural Compute Stick 2 (x 3 pieces)

According to the following article Intel engineers,
https://software.intel.com/en-us/forums/computer-vision/topic/798625

He said that we can use multiple sticks by writing below.

plugin = IEPlugin(device = "MYRIAD")
plugin.set_config({"VPU_FORCE_RESET": "NO"})

I tried it in my repository.
It operates normally.
https://github.com/PINTO0309/MobileNet-SSD-RealSense/blob/master/MultiSt.

However, connecting the three NCS 2 to the self-powered USB 3.0 hub did not improve performance at all. (9 FPS - 10 FPS)

1. What kind of implementation will I be able to use with multi stick?
2. "VPU_FORCE_RESET": "NO": Does it make sense to use?

RaspberryPi3 + NCS2 [9 FPS - 10 FPS] https://youtu.be/GJNkX-ZBuC8

Core i7 + NCS2 [21 FPS] https://youtu.be/1ogge90EuqI

0 Kudos
1 Solution
Yuanyuan_L_Intel
Employee
1,560 Views

@Hyodo, Katsuya

I wrote a very simple demo to show how to use multiple ncs2 devices for inference. You can refer to below link and there is a readme under the same folder.

https://github.com/yuanyuanli85/open_model_zoo/blob/ncs2/demos/python_demos/multiple_device_ncs2_asy...;

View solution in original post

12 Replies
Yuanyuan_L_Intel
Employee
1,560 Views

Hi, 

You can follow the "Multiple NCS Devices" section in https://software.intel.com/en-us/articles/transitioning-from-intel-movidius-neural-compute-sdk-to-op...; Basically,  multiple ExecutableNetwork instances need to be created.  Then, you can use those multiple executablenetwork instances (usually async api is used ) to accelerate your inference.

Hyodo__Katsuya
Innovator
1,560 Views
@Yuanyuan L. (Intel) Thanks for your reply. As you gave me advice, I referred to the following article. https://software.intel.com/en-us/articles/transitioning-from-intel-movidius-neural-compute-sdk-to-op... However, it contains only a very simple explanation and I do not know the implementation image. Please let me know if you have a sample program or URL that I will refer to.
Yuanyuan_L_Intel
Employee
1,561 Views

@Hyodo, Katsuya

I wrote a very simple demo to show how to use multiple ncs2 devices for inference. You can refer to below link and there is a readme under the same folder.

https://github.com/yuanyuanli85/open_model_zoo/blob/ncs2/demos/python_demos/multiple_device_ncs2_asy...;

View solution in original post

votecoffee
Beginner
535 Views
Hyodo__Katsuya
Innovator
1,560 Views
@Yuanyuan L. (Intel) Thank you for having a clear answer. I finally understood how to implement it. I will immediately begin implementing "MultiProcess" instead of "MultiThread".
Truong__Dien_Hoa
New Contributor II
1,560 Views

Sorry for the very amateur question but what is the point for using MultiProcess instead of MultiThread in this case ? I have read some tutorials on pyimagesearch about MultiThreading on video processing but not MultiProcessing. Thank you in advance

 

Hyodo__Katsuya
Innovator
1,560 Views

@Truong, Dien Hoa

It is an easy implementation like a toy, but I hope you will be helpful.

Python + MultiProcess + MultiThread + MultiModel + NCS2

https://github.com/PINTO0309/OpenVINO-EmotionRecognition.git

Python + MultiProcess + Google Edge TPU Accelerator

https://github.com/PINTO0309/TPU-MobilenetSSD.git

Truong__Dien_Hoa
New Contributor II
1,560 Views

@Katsuya, Thank you. Actually, I ask the question because passing by your code on MobileNetSSD which I think quite same concept. I am curious on why you are using MultiProcess and not just MultiThread. You are a really good practionner !! Most of things I need I can find in your work :D

Hyodo__Katsuya
Innovator
1,560 Views

Thank you for giving me a warm word. I am very happy.

In "Python + MultiThread" there is a problem of "GIL".

Global Interpreter Lock

http://www.dabeaz.com/python/UnderstandingGIL.pdf

As long as you are using Python, you can not get the best performance with multiple parallel thread handling.

Your best choice,

1. Implement MultiProcess yourself

2. Introducing Python's optional module "Flask"

 

$ sudo apt install python3-pip -y
$ pip3 install flask

 

Truong__Dien_Hoa
New Contributor II
1,560 Views

Thank you for your response. I will read carefully your document.

Nguyen__Anh_Minh
Beginner
1,560 Views

Hyodo, Katsuya wrote:

@Yuanyuan L. (Intel)

Thank you for having a clear answer.
I finally understood how to implement it.
I will immediately begin implementing "MultiProcess" instead of "MultiThread".

Hello, I find your repo OpenVINO-YoloV3 very amazing and I am checking out the openvino_yolov3_MultiStick_test.py application on my 2 sticks.

I found that you still used the multithreading for the inferencer process. should it be affected by the GIL problem too? please help me broaden my horizon on this.

thanks

Hyodo__Katsuya
Innovator
1,560 Views

@Nguyen, Anh Minh

Use NCSDK for a complete MultiProcess + MultiStick implementation.

OpenVINO can not control each stick accurately.

Reply