Show HDDL_DEVICE_ERROR when i use openvino-ep in container.

[fredchang]

I want to use HDDL to accelerator my inference by onnxruntime in container, but i got the following message. Could anyone help me to resolve it? Thanks.

2021-01-20 02:32:01.571815411 [E:onnxruntime:, sequential_executor.cc:333 Execute] Non-zero status code returned while running OpenVINO-EP-subgraph_2 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0' Status Message: /code/onnxruntime/onnxruntime/core/providers/openvino/backends/vadm_backend.cc:113 onnxruntime::openvino_ep::VADMBackend::VADMBackend(const onnxruntime::Provider_ModelProto&, onnxruntime::openvino_ep::GlobalContext&, const onnxruntime::openvino_ep::SubGraphContext&) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0_client->loadGraph(_graph, networkName, graphFileContent.data(), graphFileContent.size(), config) failed: HDDL_DEVICE_ERROR

2021-01-20 02:32:01.572340: [ONNXRuntimeError] : 6 : RUNTIME_EXCEPTION : Non-zero status code returned while running OpenVINO-EP-subgraph_2 node. Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0' Status Message: /code/onnxruntime/onnxruntime/core/providers/openvino/backends/vadm_backend.cc:113 onnxruntime::openvino_ep::VADMBackend::VADMBackend(const onnxruntime::Provider_ModelProto&, onnxruntime::openvino_ep::GlobalContext&, const onnxruntime::openvino_ep::SubGraphContext&) [OpenVINO-EP] Exception while Loading Network for graph: OpenVINOExecutionProvider_OpenVINO-EP-subgraph_2_0_client->loadGraph(_graph, networkName, graphFileContent.data(), graphFileContent.size(), config) failed: HDDL_DEVICE_ERROR

[Adli]

Hi fredchang,

Thank you for reaching out to us. We are investigating this issue and will get back to you as soon as possible.

Regards,

Adli

[fredchang]

Hi Adli,

More information to you.

The OS is Ubuntu 18.04

OpenVINO is 2021.1.110

onnxruntime is 1.6.0

There was already a device node “/dev/ion” mounted in container and a running HDDL daemon in host.

There is only one Myriad X VPU on the edge device.

Thanks.

Regards,

Fred

[Adli]

Hi fredchang,

Thank you for sharing that information. We highly value it.

Regards,

Adli

[Munesh_Intel]

Hi Fred,

Thank you for waiting and our apologies for the late reply.

For your case, we would like to isolate the issue first to find whether the cause for the error is due to HDDL component, Docker component or ONNX Runtime component.

As such, firstly, we request you to verify that HDDL plugin is working fine in Docker environment by running the Security Barrier Camera Demo, by following the steps available in either of the following links:

For built Docker image:

https://docs.openvinotoolkit.org/2021.2/openvino_docs_install_guides_installing_openvino_docker_linu...

For Ubuntu Docker image downloaded from Docker hub:

https://hub.docker.com/r/openvino/ubuntu18_dev

If the demo runs successfully and there are no issues with HDDL plugin in Docker container, then we can narrow down the cause to ONNX Runtime component. For your information, OpenVINO Execution Provider is not part of the OpenVINO toolkit, and it’s supported in ONNX Runtime GitHub by developers directly.

The instructions to run OpenVINO is given in the link below:

https://github.com/microsoft/onnxruntime/tree/master/dockerfiles#openvino

Specifically, the steps to run OpenVINO on Myriad VPU Accelerator are available here:

https://github.com/microsoft/onnxruntime/tree/master/dockerfiles#openvino-on-myriad-vpu-accelerator

If you are still facing issues after executing the steps given above, we suggest you post your questions on the ONNX Runtime GitHub (link given below) for further assistance.

https://github.com/microsoft/onnxruntime/issues

Regards,

Munesh

[Munesh_Intel]

Hi Fred,

This thread will no longer be monitored since we have provided recommendations and references. If you need any additional information from Intel, please submit a new question.

Regards,

Munesh