I am boosting with one graph(MobileNetSSD[Caffe]) + four sticks.
Is that it?
I am sorry if I made a mistake.
Basically what you need to do is give each stick its own frame. They can't work together on a frame but they can work on different frames at the same time. Each inference should take about the same amount of time so you don't have to worry about them coming out of order.
All what mail929 said is correct.
Load sharing can not be done using multiple sticks.
However, it is possible to increase the number of prediction per unit time by using multiple sticks in parallel.
Also, my sample program correctly corrects the image processing order by utilizing thread-safe Queue.
Of course, a bit of overhead will occur…
The best performance will not be obtained unless true load balancing is supported by the NCSDK API.