Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6433 Discussions

Error when doing inference with "HETERO:GPU,CPU"

MAbom
Beginner
1,651 Views

 

Hello,

 

openvino: 2021.4

I trained a mask rcnn model using tf OD API 2. I was able to use the openvino model optimizer to get the IR model.

I checked the model then using the benchmark_tool in python with both devices "CPU" and "GPU" and both worked fine.

But when trying the "Hetero" plugin I get the following exception:

[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         API version............. 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Device info
         CPU
         MKLDNNPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         GPU
         clDNNPlugin............. version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         HETERO
         heteroPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Read network took 75.03 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Network input 'image_info' precision FP32, dimensions (NC): 1 3
[ INFO ] Network input 'input_tensor' precision U8, dimensions (NCHW): 1 3 572 768
[ INFO ] Network output 'masks' precision FP32, dimensions (NCHW): 2 1 33 33
[ INFO ] Network output 'reshape_do_2d' precision FP32, dimensions (NC): 2 7
[Step 7/11] Loading the model to the device
[ ERROR ] Function contains several inputs and outputs with one friendly name!
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\main.py", line 255, in run
    exe_network = benchmark.load_network(ie_network)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\benchmark.py", line 63, in load_network
    num_requests=1 if self.api_type == 'sync' else self.nireq or 0)
  File "ie_api.pyx", line 367, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 379, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Function contains several inputs and outputs with one friendly name!

 

I ran the following command:

(tensorflow2) C:\Program Files (x86)\Intel\openvino_2021.4.582\deployment_tools\tools\benchmark_tool>python benchmark_app.py -m "C:\Users\Mouhammad\Documents\Intel\OpenVINO\openvino_models\ir\public\saved_model_smaller_FP16\saved_model.xml" -d HETERO:CPU,GPU -b 1 -api sync

 

Any idea what I am doing wrong here?

Thanks!

 

Regards,

Mouhammad

Labels (1)
0 Kudos
5 Replies
Zulkifli_Intel
Moderator
1,631 Views

Hello Mouhammad Abomoharam,

 

Greetings to you.

 

We're able to run the -d HETERO:CPU,GPU on benchmark_app using the Intel Open Model Zoo (OMZ) model. Please give it a try on the OMZ model and see whether the same error occurs or not. Share with us your model, so we can take a look at it.

 

hetero_4.PNG

 

Sincerely,

Zulkifli

 

0 Kudos
MAbom
Beginner
1,571 Views

Hello Zulkifli,

 

thanks for your reply.

 

I have downloaded and converted one of the mask rcnn models from OMZ:

https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mask_rcnn_resnet50_atrous_coco

 

I got the same error as before when using the HETERO plugin:

[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         API version............. 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Device info
         CPU
         MKLDNNPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         GPU
         clDNNPlugin............. version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         HETERO
         heteroPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Read network took 74.03 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Network input 'image_info' precision FP32, dimensions (NC): 1 3
[ INFO ] Network input 'image_tensor' precision U8, dimensions (NCHW): 1 3 800 1365
[ INFO ] Network output 'masks' precision FP32, dimensions (NCHW): 100 90 33 33
[ INFO ] Network output 'reshape_do_2d' precision FP32, dimensions (NC): 100 7
[Step 7/11] Loading the model to the device
[ ERROR ] Function contains several inputs and outputs with one friendly name!
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\main.py", line 255, in run
    exe_network = benchmark.load_network(ie_network)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\benchmark.py", line 63, in load_network
    num_requests=1 if self.api_type == 'sync' else self.nireq or 0)
  File "ie_api.pyx", line 367, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 379, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Function contains several inputs and outputs with one friendly name!

But the CPU and the GPU plugins work fine!

 

I attached also my mask rcnn model to this answer.

 

Regards,

Mouhammad

0 Kudos
Zulkifli_Intel
Moderator
1,561 Views

Hello Mouhammad Abomoharam,

 

Thank you for sharing with us the model. On our side, we also see similar behavior when using the -d HETERO:GPU,CPU argument on mask_rcnn_resnet50_atrous_coco model.

 

We noticed you also open a new thread on GitHub #6645 for a similar issue. We will continue the investigation on GitHub. Therefore, we will close this thread and focus on GitHub. Are you okay with this arrangement?

 

Sincerely,

Zulkifli

0 Kudos
MAbom
Beginner
1,558 Views

Hello Zulkifli,

 

Thanks for your reply.

 

Yes I think you can close this thread and focus on GitHub.

 

Thanks and regards,

Mouhammad

0 Kudos
Zulkifli_Intel
Moderator
1,545 Views

Hello Mouhammad Abomoharam,

 

Thank you for your feedback. This thread will no longer be monitored since this issue will continue to be discussed on Github: https://github.com/openvinotoolkit/openvino/issues/6645

 

Sincerely,

Zulkifli


0 Kudos
Reply