Retrieve inference results from intermediate layers with Intel NCS2

[vasilenko__oleg]

Hi,

I successfully use OpenVINO with Windows/CPU to get the inference results from the intermediate VGG16 layers. The part of the code that marks the intermediate layers as inference output layers is as simple as follow:

plugin = IEPlugin(device="CPU", plugin_dirs="")  # device=” MYRIAD” in case of NCS2

net = IENetwork(model=model_xml, weights=model_bin)

intermediate_layers = ['relu1_1','relu1_2','pool1','relu2_1','relu2_2','pool2','relu3_1','relu3_2',
                                    'relu3_3','pool3','relu4_1','relu4_2','relu4_3','pool4','relu5_1','relu5_2','relu5_3','pool5']
net.add_outputs(intermediate_layers )

exec_net = plugin.load(network=net)

However, when running the code on Raspbian/ARM/VPU-NCS2 (thus specifying device=” MYRIAD”) it ends up with an error:

E: [xLink] [     57547] dispatcherEventReceive:368      dispatcherEventReceive()

Read failed -1 | event 0x5e4fee20 XLINK_READ_REL_REQ

E: [xLink] [     57547] eventReader:230 eventReader stopped
E: [xLink] [     57548] XLinkReadDataWithTimeOut:1377   Event data is invalid
E: [ncAPI] [     57548] ncGraphAllocate:1784    Can't read input tensor descriptors of the graph, rc: X_LINK_ERROR
Traceback (most recent call last):
  File "vgg16_openvino.py", line 47, in <module>
    exec_net = plugin.load(network=net)
  File "ie_api.pyx", line 395, in openvino.inference_engine.ie_api.IEPlugin.load
  File "ie_api.pyx", line 406, in openvino.inference_engine.ie_api.IEPlugin.load
RuntimeError: Failed to allocate graph: NC_ERROR

To note, if I don’t add the intermediate output layers, NCS2 runs the code without an error.

Could you please give a hint on how to solve the problem of getting inference outputs for the intermediate layers with NCS2?

Thanks in advance,

Oleg

[Shubha_R_Intel]

Dear vasilenko, oleg,

I answered this very question in an older forum post . The key is to use network.addOutput(). I asked the poster to refer to object_detection_demo for an example but unfortunately that sample doesn't exist anymore in OpenVino 2019R2. The same technique exists however in demos\mask_rcnn_demo\main.cpp which is available in OpenVino 2019R2.

Hope it helps,

Thanks,

Shubha