Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

HETERO Target Device for OpenVino 2020.4

Mauricio
Novice
1,644 Views

Hi,

I have tried to set the target device as HETERO with the C++ and Python examples, but it doesn't work.

I have followed the most basic configuration in this guide:
https://docs.openvinotoolkit.org/2020.4/openvino_docs_IE_DG_supported_plugins_HETERO.html

I'm using OpenVino 2020.4, FP32 models.

They (CPU,GPU,MYRIAD) work individually but not together.

How can we use the HETERO configuration? , like:
HETERO:CPU,GPU
HETERO:CPU:MYRIAD
HETERO:MYRIAD:GPU

Thanks.

0 Kudos
1 Solution
Peh_Intel
Moderator
1,591 Views

Hi Mauricio,

 

Greetings to you.

 

If you want to configure a default inference device as HETERO instead of CPU, you need to modify the Python code accordingly.

 

I would still strongly suggest you to use command line for the HETERO configuration. The heterogeneous plugin run the network layers on multiple device according to specified priority. The first device will be the first priority device, if a specific layer is not supported for this device, the hetero plug-in will fall back into the second priority device and so on.

 

I’ve checked the safety gear detector demo. The model used in this demo, person-detection-retail-0013 model, supports CPU,GPU,MYRIAD and also HETERO:FPGA,CPU. You may find in here.

As such, it doesn’t make sense performance wise to run them in these configuration ways : HETERO:CPU,GPU, HETERO:CPU,MYRIAD, HETERO:MYRIAD,GPU

 

Please note that some of the topologies are not friendly to heterogeneous execution on some devices or cannot be executed in such mode at all or it makes no sense performance wise to run them in this way.

 

Regards,

Peh

 

View solution in original post

0 Kudos
5 Replies
Peh_Intel
Moderator
1,629 Views

Hi Mauricio,

Greetings to you.

I strongly recommend you use command line for HETERO configuration. An example implementation for Image Classification Python Sample Async sample is shown below:

python <path_to_openvino>\deployment_tools\inference_engine\samples\python\classification_sample_async\classification_sample_async.py -i <path_to_image>\cat.bmp -m <path_to_model>\alexnet_fp32.xml -nt 5 -d HETERO:CPU,GPU


However, if you still desire to change the target device in the codes directly, please specify which example you are trying for me to better assist you.


Regards,

Peh


0 Kudos
Mauricio
Novice
1,617 Views

Hello again,

We need to edit the device from the code.

We are using OpenVino's safety_gear example (python) on Intel's EIS framework. (Edge Insight for Industrial).

Regards.

0 Kudos
Peh_Intel
Moderator
1,592 Views

Hi Mauricio,

 

Greetings to you.

 

If you want to configure a default inference device as HETERO instead of CPU, you need to modify the Python code accordingly.

 

I would still strongly suggest you to use command line for the HETERO configuration. The heterogeneous plugin run the network layers on multiple device according to specified priority. The first device will be the first priority device, if a specific layer is not supported for this device, the hetero plug-in will fall back into the second priority device and so on.

 

I’ve checked the safety gear detector demo. The model used in this demo, person-detection-retail-0013 model, supports CPU,GPU,MYRIAD and also HETERO:FPGA,CPU. You may find in here.

As such, it doesn’t make sense performance wise to run them in these configuration ways : HETERO:CPU,GPU, HETERO:CPU,MYRIAD, HETERO:MYRIAD,GPU

 

Please note that some of the topologies are not friendly to heterogeneous execution on some devices or cannot be executed in such mode at all or it makes no sense performance wise to run them in this way.

 

Regards,

Peh

 

0 Kudos
Mauricio
Novice
1,561 Views

Thank you very much for your reply.

I thought that a better performance could be obtained this way. But I understand that maybe it doesn't make much sense.

I think that's enough for now.

Best regards!

0 Kudos
Peh_Intel
Moderator
1,533 Views

Hi Mauricio,

 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

 

Regards,

Peh


0 Kudos
Reply