Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Question about use multi sticks

tang__chu
Beginner
738 Views

 

Hello:
I have two sticks, one is Intel neural compute stick and another is Intel neural compute stick 2, I want to use multi sticks to inference models. The first situation is to use two sticks to infer one CNN model, the operation system dispatches the computation burden to each stick automatically.
Another situation is to infer a complete CNN model on each stick respectively, for example, use one stick to inference model1, use another stick to inference model2, can I manually dispatch this mission to each stick?
Can these two assumptions be realized and how to do that in python? Can you offer some sample codes?
In the Openvino samples, I only found python codes to infer one model on one stick.

0 Kudos
10 Replies
Yuanyuan_L_Intel
Employee
738 Views

Hi, TangChu

The two use cases can be supported by OpenVino. Basically,  multiple ExecutableNetwork instances need to be created, and one instance for each device.  Though, there is nothing in the API that ties an ExecutableNetwork to a particular device.

For python sample code, you can referto this repo multiple_device_ncs2_async.py  which shows how to run one CNN model in multiple ncs2 via multiple threads.

 

 

0 Kudos
tang__chu
Beginner
738 Views

thanks Yuanyuan, but I still have a question, can python infer multiple models in the openvino? or just can write C++ to infer multiple models in the openvino?

0 Kudos
tang__chu
Beginner
738 Views

hi, Yuanyuan, I couldn't open this URL your given"multiple_device_ncs2_async.py," if you are convenient, could you send the sample code "multiple_device_ncs2_async.py" to my e-mail, my email is: chu.tang@anu.edu.au. thanks again.

0 Kudos
tang__chu
Beginner
738 Views

hi, Yuanyuan, I have found the"multiple_device_ncs2_async.py" in github, thanks, just wondering can Python infer multiple models in the openvino?

0 Kudos
Yuanyuan_L_Intel
Employee
738 Views

@tang, chu

 > , just wondering can Python infer multiple models in the openvino?

Yes, OpenVino can infer multiple models on same device, or infer one models on one device.  If you infer multiple models on one ncs2, it may hit the memory limitation.

0 Kudos
tang__chu
Beginner
738 Views

thanks Yuanyuan, now I want to use Python to infer multiple models, I have looked all python samples in the openvino, but all samples just infer one model, where I can find the Python sample to infer multiple models?

0 Kudos
Yuanyuan_L_Intel
Employee
738 Views

@tang, chu 

There is not too much difference b/w c++ and python interface to infer multiple models.  For c++, you can take a look at crossroad_camera_demo.hpp for details.

0 Kudos
tang__chu
Beginner
738 Views

ok, thanks yuanyuan.

0 Kudos
Kulecz__Walter
New Contributor I
738 Views

tang, chu wrote:

hi, Yuanyuan, I have found the"multiple_device_ncs2_async.py" in github, thanks, just wondering can Python infer multiple models in the openvino?

 

Where did you find it?  I'm coming up empty searching github and Google

 

0 Kudos
Reply