Chaining Movidius Sticks

idata · ‎07-25-2017

I would like to chain several Movidus Sticks together to increase throughput. Do I need any special hardware, or will a standard USB 3 hub be sufficient?

idata · ‎07-25-2017

Jessica are you thinking of doing RCNN by sending each region to a stick for inference?

idata · ‎07-25-2017

I saw a picture of a 4-headed USB hub with four of the sticks plugged into it on one of the websites, but I don't remember which one.

idata · ‎07-26-2017

Hi @Jessica, @chicagobob123, @znmeb

No special hardware is required to use multiple NCS devices. A standard USB hub will work for this. Your software will need to handle each device by feeding it input and retrieving inferences from it.

Neal

idata · ‎07-26-2017

@neal thanks

idata · ‎07-26-2017

I saw the picture last night someplace as well. 4 connected to a single hub.

They reported that they worked together, like it was magic, but left no details on how.

idata · ‎07-27-2017

@chicagobob123 the concept of scaling is quite simple - every device that's plugged into your host machine gets a zero-index unique ID, which can be used to reference a specific stick while calling any of the MvNC APIs. Use EnumerateDevices function to get a list of devices, which can be used with the Device class to generate a handle. Here's a code snippet to help you get started:

from mvnc import mvncapi as mvnc devices = mvnc.EnumerateDevices() devHandle.append(mvnc.Device(devices[devnum])) devHandle[devnum].OpenDevice()

You can now use devHandle to perform any operations on a specific device.

idata · ‎08-24-2017

Hi@AshwinVijayakumar

In order to use 4 movidius sticks at the same time, I changed a part of the code of classification_example.py as follows.

devHandle.append(mvnc.Device(devices[0]))
devHandle.append(mvnc.Device(devices[1]))
devHandle.append(mvnc.Device(devices[2]))
devHandle.append(mvnc.Device(devices[3]))
devHandle[0].OpenDevice()
devHandle[1].OpenDevice()
devHandle[2].OpenDevice()
devHandle[3].OpenDevice()

However, inference time does not change.

Is this code correct? Please give me some advices.

Thanks.

hide

idata · ‎08-24-2017

Can you please paste your complete code here. The trick is to send four images, one to each stick using LoadTensor and then trying to get the result back from each stick using GetResult()

Hope this code below helps

def runparallel(count=100, num=[]): 
     numdevices = num 
     if len(num) == 0: numdevices = range(len(devices)) 

      for i in range(count): 
         # ***************************************************************** 
         # Load the Tensor to each of the devices 
         # ***************************************************************** 
         for devnum in numdevices: 
             img = choice(imgarr) 
             graphHandle[devnum].LoadTensor(img, 'user object') 

         # ***************************************************************** 
         # Read the result from each of the devices 
         # ***************************************************************** 
         for devnum in numdevices: 
             tensor, userobj = graphHandle[devnum].GetResult()

idata · ‎08-25-2017

Thanks @ramana.rachakonda

Is it possible to use 4 sticks for inference of just 1 image in parallel as GPU does?

idata · ‎08-25-2017

No, not at this time You can send four different images to each of the four sticks. Or you can load different networks on each stick and run them independently.

idata · ‎08-25-2017

@ramana.rachakonda

Oh, I see.Thank you for the nice advice.

idata · ‎12-29-2017

I have opened a pull request at: https://github.com/duangenquan/YoloV2NCS/pull/6

Enable multiple NCS devices to improve the throughput, hope this helps.

idata · ‎12-30-2017

My previous pull request was merged, visit this link for details: https://github.com/duangenquan/YoloV2NCS