Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

HETERO Target Device for OpenVino 2020.4

Mauricio
Novice
465 Views

Hi,

I have tried to set the target device as HETERO with the C++ and Python examples, but it doesn't work.

I have followed the most basic configuration in this guide:
https://docs.openvinotoolkit.org/2020.4/openvino_docs_IE_DG_supported_plugins_HETERO.html

I'm using OpenVino 2020.4, FP32 models.

They (CPU,GPU,MYRIAD) work individually but not together.

How can we use the HETERO configuration? , like:
HETERO:CPU,GPU
HETERO:CPU:MYRIAD
HETERO:MYRIAD:GPU

Thanks.

0 Kudos
1 Solution
Peh_Intel
Moderator
412 Views

Hi Mauricio,

 

Greetings to you.

 

If you want to configure a default inference device as HETERO instead of CPU, you need to modify the Python code accordingly.

 

I would still strongly suggest you to use command line for the HETERO configuration. The heterogeneous plugin run the network layers on multiple device according to specified priority. The first device will be the first priority device, if a specific layer is not supported for this device, the hetero plug-in will fall back into the second priority device and so on.

 

I’ve checked the safety gear detector demo. The model used in this demo, person-detection-retail-0013 model, supports CPU,GPU,MYRIAD and also HETERO:FPGA,CPU. You may find in here.

As such, it doesn’t make sense performance wise to run them in these configuration ways : HETERO:CPU,GPU, HETERO:CPU,MYRIAD, HETERO:MYRIAD,GPU

 

Please note that some of the topologies are not friendly to heterogeneous execution on some devices or cannot be executed in such mode at all or it makes no sense performance wise to run them in this way.

 

Regards,

Peh

 

View solution in original post

5 Replies
Peh_Intel
Moderator
450 Views

Hi Mauricio,

Greetings to you.

I strongly recommend you use command line for HETERO configuration. An example implementation for Image Classification Python Sample Async sample is shown below:

python <path_to_openvino>\deployment_tools\inference_engine\samples\python\classification_sample_async\classification_sample_async.py -i <path_to_image>\cat.bmp -m <path_to_model>\alexnet_fp32.xml -nt 5 -d HETERO:CPU,GPU


However, if you still desire to change the target device in the codes directly, please specify which example you are trying for me to better assist you.


Regards,

Peh


Mauricio
Novice
438 Views

Hello again,

We need to edit the device from the code.

We are using OpenVino's safety_gear example (python) on Intel's EIS framework. (Edge Insight for Industrial).

Regards.

Peh_Intel
Moderator
413 Views

Hi Mauricio,

 

Greetings to you.

 

If you want to configure a default inference device as HETERO instead of CPU, you need to modify the Python code accordingly.

 

I would still strongly suggest you to use command line for the HETERO configuration. The heterogeneous plugin run the network layers on multiple device according to specified priority. The first device will be the first priority device, if a specific layer is not supported for this device, the hetero plug-in will fall back into the second priority device and so on.

 

I’ve checked the safety gear detector demo. The model used in this demo, person-detection-retail-0013 model, supports CPU,GPU,MYRIAD and also HETERO:FPGA,CPU. You may find in here.

As such, it doesn’t make sense performance wise to run them in these configuration ways : HETERO:CPU,GPU, HETERO:CPU,MYRIAD, HETERO:MYRIAD,GPU

 

Please note that some of the topologies are not friendly to heterogeneous execution on some devices or cannot be executed in such mode at all or it makes no sense performance wise to run them in this way.

 

Regards,

Peh

 

View solution in original post

Mauricio
Novice
382 Views

Thank you very much for your reply.

I thought that a better performance could be obtained this way. But I understand that maybe it doesn't make much sense.

I think that's enough for now.

Best regards!

Peh_Intel
Moderator
354 Views

Hi Mauricio,

 

This thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

 

Regards,

Peh


Reply