How to use multi Myriad sticks on Raspberry Pi

[Saw__Joo]

Hi,

I installed OpenVino 2019 R3 (openvino_toolkit_runtime_raspbian_p_2019.3.334.tgz) on Raspbian. Single Myriad stick works fine, but when I try to run with multi-stick, I get missing libMultiDevicePlugin.so error. I searched the installed directory but can't find the library. Where can I find this library?

I also tried instantiating "executable_network = ie.LoadNetwork(network, one_myriad_stick, { })" once per stick, each time pointing it to a different Myriad stick like MYRIAD.1.2.1-ma2480, or MYRIAD.1.2.3-ma2480, but I get the below error when I try to do inference.

E: [xLink] [    974617] [EventRead00Thr] eventReader:218        eventReader thread stopped (err -4)
E: [global] [    974617] [Scheduler00Thr] dispatcherEventSend:1004      Write failed (header) (err -4) | event XLINK_WRITE_REQ

E: [xLink] [    974619] [Scheduler00Thr] eventSchedulerRun:626  Event sending failed
E: [xLink] [    974620] [Scheduler00Thr] eventSchedulerRun:576  Dispatcher received NULL event!

Question is, how do I get multi-stick to work?

Any help is greatly appreciated.

[JesusE_Intel]

Hi Joo,

The installation instructions mention "The OpenVINO™ toolkit for Raspbian* OS includes the Inference Engine and the MYRIAD plugins". It doesn't look like the MULTI Device plugin is included in the OpenVINO™ Toolkit for Raspbian OS.

I am reaching out to the development team for confirmation, I will let you know what I find out.

Regards,

Jesus

[Saw__Joo]

@JesusE Thank you for looking into the missing library issue. While waiting for a response from the development team, can you please help me with my 2nd method of using multi-myriad sticks?

From other threads, it looks like it should be possible to instantiate multiple executable networks from ie.LoadNetwork, one for each stick, by passing the myriad device name as an argument, such as MYRIAD.1.2.1-ma2480 and MYRIAD.1.2.3-ma2480, but I keep getting this error:

E: [xLink] [    732879] [EventRead00Thr] eventReader:218        eventReader thread stopped (err -4)
E: [xLink] [    732879] [Scheduler00Thr] eventSchedulerRun:576  Dispatcher received NULL event!
E: [global] [    732879] [human_pose_esto] XLinkReadDataWithTimeOut:1494       Event data is invalid
E: [ncAPI] [    732879] [human_pose_esto] ncFifoReadElem:3510   Packet reading is failed.
E: [xLink] [    733077] [EventRead01Thr] eventReader:218        eventReader thread stopped (err -4)
E: [xLink] [    733077] [Scheduler01Thr] eventSchedulerRun:576  Dispatcher received NULL event!
E: [watchdog] [    733104] [WatchdogThread] sendPingMessage:121 Failed send ping message: X_LINK_ERROR

This is what I'm doing in my C++ code:

Core ie;
std::vector<std::string> availableDevices = ie.GetAvailableDevices();
CNNNetReader netReader;
netReader.ReadNetwork(modelXmlPath);
netReader.ReadWeights(modelBinPath);

CNNNetwork network = netReader.getNetwork();
network.setBatchSize(1);

InputsDataMap input_data_map = network.getInputsInfo();
InputInfo::Ptr inputInfo = input_data_map.begin()->second;
inputInfo->setPrecision(Precision::U8);

// I have 3 myriad sticks, so hard-code to 3 for simplicity
// It works if I change 3 to 1
ExecutableNetwork executableNetwork[3];
InferRequest async_infer_request[3];

for (int i = 0; i < 3; i++) {
    executableNetwork = ie.LoadNetwork(network, availableDevices.at(i), {});
    async_infer_request = executableNetwork.CreateInferRequest();
}

// Get input image
cv::Mat image = getInputImage();
// Pre-process input
preprocess_input(&async_infer_request[0], image);
// Start async inference
async_infer_request[0].StartAsync();

Can you please tell me what I'm doing wrong?

[JesusE_Intel]

Hi Joo,

The development team confirmed that the Multi-Device plugin is not included in the Raspberry Pi package, it is only available for the full version of the OpenVINO(tm) Toolkit. If I'm not mistaken, you don't have to specify the device name but a device ID (0,1,2). I have not tried this myself but take a look at the following Python sample we wrote, it should be similar in C++.

https://github.com/movidius/ncappzoo/blob/master/apps/benchmark_ncs/benchmark_ncs.py

Hope this helps!

Regards,

Jesus

[Saw__Joo]

@JesusE I don't see a way of specifying device ID in the LoadNetwork function in C++.

This is the API from the documentation:

InferenceEngine::Core::LoadNetwork(CNNNetwork network, const std::string & deviceName, const std::map< std::string, std::string > & config = std::map< std::string, std::string >())

I tried ie.LoadNetwork(network, 0) and as expected, it does not compile. I tried calling ie.LoadNetwork(network, "MYRIAD") once per myriad stick, but it gives me the exact same error as before.

As a summary of what I've tried:

1. ie.LoadNetwork(network, "MULTI", {}) does not work because Intel omits libMultiDevicePlugin.so. When is Intel going to include this in the Raspbian release?
2. call ie.LoadNetwork(network, "MYRIAD") multi times, once per myriad stick, does not work. The Openvino framework gives me eventSchedulerRun error. Works fine with one myriad stick.
3. call ie.LoadNetwork(network, availableDevices.at(i)) once per myriad stick does not work. The Openvino framework gives me eventSchedulerRun error. Works fine with one myriad stick.
4. call ie.LoadNetwork(network, device_ID_as_int) as suggested does not compile.
5. Run 2 independent programs. Both programs do exactly the same thing, except program A is hard-coded to use MYRIAD.1.2.1-ma2480 device only, and program B is hard-coded to use MYRIAD.1.2.3-ma2480 device only. Both program A and program B run perfectly when it is the only program running in the environment. While program A is running, if I fire up program B, I get the eventSchedulerRun error from OpenVino on both programs.

I don't know what else to try. Please help.

[JesusE_Intel]

Hi Joo,

Thank you for your patience while I looked into this. It seems your code is doing it the right way. I was able to use two Intel NCS 2 sticks for inference by the same application on Ubuntu running a the full version of OpenVINO(tm) toolkit. Using the same code on the Pi gave me a segmentation fault when creating the second ExecutableNetwork. This may be a bug and I will need to confirm with the development team.

ExecutableNetwork executableNetwork[2];
InferRequest async_infer_request[12];

executableNetwork[0] = ie.LoadNetwork(network, "MYRIAD", {});
executableNetwork[1] = ie.LoadNetwork(network, "MYRIAD", {});

for (int i = 0; i < 12; i++) {
	async_infer_request.StartAsync();
}
for (int i = 0; i < 12; i++){
    async_infer_request.Wait(IInferRequest::WaitMode::RESULT_READY);
}

Also, the XLINK error is usually seen when the device is not getting enough power. Are you using a powered USB hub to connect the sticks to the Raspberry Pi? 

Regards,

Jesus

[Saw__Joo]

@JesusE Thank you for explaining the XLINK error message. I moved all my NCS2 sticks to a powered hub, made a few changes to my code and managed to run multiple sticks in parallel. I am calling ie.LoadNetwork(network, availableDevices.at(i), {}) rather than just MYRIAD string.

I have a few questions regarding the openvino framework (I can't seem find any documentation on the behaviour):

1. Inference done callback (registered via SetCompletionCallback) - Calling async_infer_request.Wait(IInferRequest::WaitMode::STATUS_ONLY) within the callback context gives me status -9 which is RESULT_NOT_READY. Question is, how do I guarantee that the result is ready before I read it out?
2. Calling ie.LoadNetwork in parallel (using std::async) gives me this error: "E: [ncAPI] [    514791] [human_pose_esti] ncGraphAllocate:1862  The graph file is bigger than the device memory". Why?
3. How much power does the NCS2 consume? I've seen 1W being thrown around, but can't find any numbers in the product datasheet. Now we know it is definitely higher than 1W. Question is, what is the maximum power consumption of an NCS2?