Issue running 2 networks on NCS2

[Gouveia__César]

Hi guys,

My issue happens when I try to run two networks (a face detector from your model zoo plus a landmark extractor from your model zoo too) using the same NCS2. My two previous tests were to run your landmark extractor only on NCS2. And after that your face detector only on NCS2 too. And they both got sucessfull. However when I try to run the 2 networks at the same time I got this error:

Can not init Myriad device: NC_ERROR
..\src\vpu\myriad_plugin\myriad_executor.cpp:251

So line 251 of the myriad executor says:

if (status != NC_OK) {
        THROW_IE_EXCEPTION << "Failed to allocate graph: " << ncStatusToStr(nullptr, status);
}

Given that my pipeline starts with the face detector and then goes to the face landmark extractor, I think it is saying that it can't allocate the landmark extractor graph. So this leads me to a couple of questions:

• It is possible to run more than one model/network on a single NCS2?
• Do I have to do anything before reading the next model? Like, clean up the "movidius cache" with some sort of openvino instruction? 

My environment goes like this:

•  Windows 10
• OpenVINO 2019.2
• NCS2
• Face detector model from model zoo: https://download.01.org/opencv/2019/open_model_zoo/R1/20190404_140900_models_bin/face-detection-retail-0004/FP16/
• Face landmark extractor from model zoo: https://download.01.org/opencv/2019/open_model_zoo/R1/20190404_140900_models_bin/facial-landmarks-35-adas-0002/FP16/

[Gouveia__César]

bump

[RandallMan_B_Intel]

Hi César, 

What command are you using?  

Yes, you can run two models simultaneously with only one NCS2, with lower performance result of course. We have several samples that handle multiple models like the Interactive Face Detection C++ Demo. 

I think the below threads will help you. 

• https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit/topic/805977
• https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit/topic/810186

Best regards, 

Randall B.

[Gouveia__César]

Hi Randall, thank you very much for your response!

So I'm not using any commands because I'm not using the OpenVINO Toolkit, I'm just using the InferenceEngine in my application by following those steps:

https://docs.openvinotoolkit.org/latest/_docs_IE_DG_Integrate_with_customer_application_new_API.html

My solution works perfectly fine when I use the CPU Plugin, and I run two networks without a problem, for example the face detector and the landmark extractor. However, when I change the TargetDevice for Myriad instead of CPU (basically using the MyriadPlugin instead of the MKLDNNPlugin - CPU) I get the error described above.

The only thing that I changed in the code was the dispatcher line that loads the target plugin:

dispatcher.getPluginByName("myriadPlugin")

instead of 

dispatcher.getPluginByName("MKLDNNPlugin")

Do I need to change anything else besides that?

I can send you a diagram showing the flow that I used in my code if that helps. The version of OpenVINO that I'm using in my solution is the R2.

[Gouveia__César]

I have made a simple diagram (see attached figure) which maps the flow of my solution when I run your face detector model and face landmark extractor models/networks sequentially. Basically I'm repeating the whole pipeline and I think this might be the problem. However if I run this type of flow with the CPU plugin it runs smoothly, which is very strange. Why this type of pipeline works on CPU (CPU plugin) but not on MYRIAD (Myriad Plugin)??

[Sahira_Intel]

Hi Cesar,

You should not need to change anything besides the target device. 

Is your script recognizing that target device? Can you include the GetAvailableDevices() function in your script? It should return a list of devices. Verify that Myriad is on there and is a recognized target device.

Best Regards,

Sahira