Inference Engine Runtime Error on NCS2 vs. CPU

Vici · ‎01-11-2019

Dear all,

I am having issues deploying a frozen graph, successfully that converted into IR model (xml, bin), to the NCS2.

Running on Windows 10, Python 3.6.5, using the OpenVINO Python API. The sample code is minimal (skipping non relevant parts):

from openvino.inference_engine import IENetwork, IEPlugin

def main():
    #######################  Device  Initialization  ########################
    #  Plugin initialization for specified device and load extensions library if specified
    plugin = IEPlugin(device="MYRIAD")
    # plugin = IEPlugin(device="GPU")
    
    #  Read in Graph file (IR)
    net = IENetwork(model="sample16.xml", weights="sample16.bin")

    print(net.inputs.keys())
    print(net.outputs)

    log.info("Preparing input blobs")
    input_blob = next(iter(net.inputs))
    out_blob = next(iter(net.outputs))

    # Read and pre-process input images
    print(net.inputs[input_blob].shape)
    print(net.outputs[out_blob].shape)

    #  Load network to the plugin
    exec_net = plugin.load(network=net)
    ### END OF SAMPLE ###

When I call plugin.load with the same network description (FP16 converted with the Model Optimizer successsfully), I get different responses.

Using the GPU plugin (CPU also runs fine with FP32 version of same protobuf file):

GPU Plugin

NCS2_Sample>python sample.py
dict_keys(['input_2'])
{'add_2/add': <openvino.inference_engine.ie_api.OutputInfo object at 0x00000193935EDB70>, 'lambda_2/strided_slice/Split.1': <openvino.inference_engine.ie_api.OutputInfo object at 0x00000193935EDBE8>}
[1, 3, 240, 320]
[1, 1, 240, 320]
loaded

Runs successfully up to loading the network.

Using the MYRIAD plugin:

NCS2 Plugin

NCS2_Sample>python sample.py
dict_keys(['input_2'])
{'add_2/add': <openvino.inference_engine.ie_api.OutputInfo object at 0x000002850F48DB70>, 'lambda_2/strided_slice/Split.1': <openvino.inference_engine.ie_api.OutputInfo object at 0x000002850F48DBE8>}
[1, 3, 240, 320]
[1, 1, 240, 320]
Traceback (most recent call last):
File "sample.py", line 86, in <module>
sys.exit(main() or 0)
File "sample.py", line 46, in main
exec_net = plugin.load(network=net)
File "ie_api.pyx", line 389, in openvino.inference_engine.ie_api.IEPlugin.load
File "ie_api.pyx", line 400, in openvino.inference_engine.ie_api.IEPlugin.load
RuntimeError: AssertionFailed: output->desc().dimsOrder() == inDesc.dimsOrder()

There is something up with the dimensions or their ordering, I suppose, but I have checked above the input and output dimensions and they are quite correct. Also, that does not explain why the network loads on CPU and GPU without problems, while MYRIAD returns a runtime error. How can I debug this issue?

Jun__Wen · ‎06-02-2019

I have the same problem as you. Can you solve this problem?

E: [xLink] [ 158539] dispatcherEventSend:924 Write failed header -7 | event XLINK_WRITE_REQ

E: [xLink] [ 158539] eventSchedulerRun:584 Event sending failed
E: [xLink] [ 159902] dispatcherEventReceive:347 dispatcherEventReceive() Read failed -1 | event 0x7ff30fffeee0 XLINK_WRITE_REQ

E: [xLink] [ 159902] eventReader:233   eventReader stopped
E: [watchdog] [ 159903] sendPingMessage:132   Failed send ping message: X_LINK_ERROR
E: [xLink] [ 159903] XLinkReadDataWithTimeOut:1343   Event data is invalid
E: [ncAPI] [ 159903] ncGraphAllocate:1828   Can't read output tensor descriptors of the graph, rc: X_LINK_ERROR
Traceback (most recent call last):
File "run.py", line 293, in <module>
sys.exit(main() or 0)
File "run.py", line 204, in main
exec_net = plugin.load(network=net)
File "ie_api.pyx", line 395, in openvino.inference_engine.ie_api.IEPlugin.load
File "ie_api.pyx", line 406, in openvino.inference_engine.ie_api.IEPlugin.load
RuntimeError: Failed to allocate graph: NC_ERROR

Shubha_R_Intel · ‎06-05-2019

Dear Vici,

First, you didn't say much about your model. Is it a custom model ? Please make sure that your layers are supported:

http://docs.openvinotoolkit.org/latest/_docs_IE_DG_supported_plugins_Supported_Devices.html

If your layers are supported, then it could be an OpenVino bug. Rather than go through all the trouble to debug, rather I'd prefer you attach a zip file with your model to this ticket and let me investigate it. If you'd rather not publicize your model let me know and I will PM you so that you can send it to me privately.

You can debug it as follows.

Download the OpenVino dldt opensource:

https://github.com/opencv/dldt

Build a debug version of Inference Engine, by following this README:

https://github.com/opencv/dldt/blob/2019/inference-engine/README.md

Build a debug version of the VPU plugin which has been completely open-sourced too by studying this:

https://github.com/opencv/dldt/tree/2019/inference-engine/src/vpu/myriad_plugin

Please make sure to regenerate your IR using DLDT Open Source too (don't use your existing IR).

By stepping into IE and VPU source code, you will narrow down the issue.

Thanks,

Shubha

V_Marco · ‎11-20-2019

I have the same issue, i used the hetero plugin to know which layer gives me the error and is this one:

this layer is generated automatically by the model optimizer and when i assign affinity (MYRIAD) to this layer it gives me this error:
RuntimeError: AssertionFailed: !onlyUsedOutputs.empty()

and if i use only MYRIAD PLUGIN it gaves me this error:
RuntimeError: AssertionFailed: _allocatedIntermData.count(topParent) > 0

i also checked if it's supported and is supported can you give me some tips to understand how i can investigate the problem?