HETERO Target Device for OpenVino 2020.4

[Mauricio]

Hi,

I have tried to set the target device as HETERO with the C++ and Python examples, but it doesn't work.

I have followed the most basic configuration in this guide:
https://docs.openvinotoolkit.org/2020.4/openvino_docs_IE_DG_supported_plugins_HETERO.html

I'm using OpenVino 2020.4, FP32 models.

They (CPU,GPU,MYRIAD) work individually but not together.

How can we use the HETERO configuration? , like:
HETERO:CPU,GPU
HETERO:CPU:MYRIAD
HETERO:MYRIAD:GPU

Thanks.

[Peh_Intel]

Hi Mauricio,

Greetings to you.

I strongly recommend you use command line for HETERO configuration. An example implementation for Image Classification Python Sample Async sample is shown below:

python <path_to_openvino>\deployment_tools\inference_engine\samples\python\classification_sample_async\classification_sample_async.py -i <path_to_image>\cat.bmp -m <path_to_model>\alexnet_fp32.xml -nt 5 -d HETERO:CPU,GPU

However, if you still desire to change the target device in the codes directly, please specify which example you are trying for me to better assist you.

Regards,

Peh

[Mauricio]

Hello again,

We need to edit the device from the code.

We are using OpenVino's safety_gear example (python) on Intel's EIS framework. (Edge Insight for Industrial).

Regards.

[Peh_Intel · Accepted solution]

Hi Mauricio,

Greetings to you.

If you want to configure a default inference device as HETERO instead of CPU, you need to modify the Python code accordingly.

I would still strongly suggest you to use command line for the HETERO configuration. The heterogeneous plugin run the network layers on multiple device according to specified priority. The first device will be the first priority device, if a specific layer is not supported for this device, the hetero plug-in will fall back into the second priority device and so on.

I’ve checked the safety gear detector demo. The model used in this demo, person-detection-retail-0013 model, supports CPU,GPU,MYRIAD and also HETERO:FPGA,CPU. You may find in here.

As such, it doesn’t make sense performance wise to run them in these configuration ways : HETERO:CPU,GPU, HETERO:CPU,MYRIAD, HETERO:MYRIAD,GPU

Please note that some of the topologies are not friendly to heterogeneous execution on some devices or cannot be executed in such mode at all or it makes no sense performance wise to run them in this way.

Regards,

Peh

[Mauricio]

Thank you very much for your reply.

I thought that a better performance could be obtained this way. But I understand that maybe it doesn't make much sense.

I think that's enough for now.

Best regards!

[Peh_Intel]

Hi Mauricio,

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Peh