Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5841 Discussions

Error when doing inference with "HETERO:GPU,CPU"

MAbom
Beginner
626 Views

 

Hello,

 

openvino: 2021.4

I trained a mask rcnn model using tf OD API 2. I was able to use the openvino model optimizer to get the IR model.

I checked the model then using the benchmark_tool in python with both devices "CPU" and "GPU" and both worked fine.

But when trying the "Hetero" plugin I get the following exception:

[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         API version............. 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Device info
         CPU
         MKLDNNPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         GPU
         clDNNPlugin............. version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         HETERO
         heteroPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Read network took 75.03 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Network input 'image_info' precision FP32, dimensions (NC): 1 3
[ INFO ] Network input 'input_tensor' precision U8, dimensions (NCHW): 1 3 572 768
[ INFO ] Network output 'masks' precision FP32, dimensions (NCHW): 2 1 33 33
[ INFO ] Network output 'reshape_do_2d' precision FP32, dimensions (NC): 2 7
[Step 7/11] Loading the model to the device
[ ERROR ] Function contains several inputs and outputs with one friendly name!
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\main.py", line 255, in run
    exe_network = benchmark.load_network(ie_network)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\benchmark.py", line 63, in load_network
    num_requests=1 if self.api_type == 'sync' else self.nireq or 0)
  File "ie_api.pyx", line 367, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 379, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Function contains several inputs and outputs with one friendly name!

 

I ran the following command:

(tensorflow2) C:\Program Files (x86)\Intel\openvino_2021.4.582\deployment_tools\tools\benchmark_tool>python benchmark_app.py -m "C:\Users\Mouhammad\Documents\Intel\OpenVINO\openvino_models\ir\public\saved_model_smaller_FP16\saved_model.xml" -d HETERO:CPU,GPU -b 1 -api sync

 

Any idea what I am doing wrong here?

Thanks!

 

Regards,

Mouhammad

Labels (1)
0 Kudos
5 Replies
Zulkifli_Intel
Moderator
606 Views

Hello Mouhammad Abomoharam,

 

Greetings to you.

 

We're able to run the -d HETERO:CPU,GPU on benchmark_app using the Intel Open Model Zoo (OMZ) model. Please give it a try on the OMZ model and see whether the same error occurs or not. Share with us your model, so we can take a look at it.

 

hetero_4.PNG

 

Sincerely,

Zulkifli

 

MAbom
Beginner
546 Views

Hello Zulkifli,

 

thanks for your reply.

 

I have downloaded and converted one of the mask rcnn models from OMZ:

https://github.com/openvinotoolkit/open_model_zoo/tree/master/models/public/mask_rcnn_resnet50_atrou...

 

I got the same error as before when using the HETERO plugin:

[Step 1/11] Parsing and validating input arguments
[ WARNING ]  -nstreams default value is determined automatically for a device. Although the automatic selection usually provides a reasonable performance, but it still may be non-optimal for some cases, for more information look at README.
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
         API version............. 2021.4.0-3839-cd81789d294-releases/2021/4
[ INFO ] Device info
         CPU
         MKLDNNPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         GPU
         clDNNPlugin............. version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4
         HETERO
         heteroPlugin............ version 2.1
         Build................... 2021.4.0-3839-cd81789d294-releases/2021/4

[Step 3/11] Setting device configuration
[Step 4/11] Reading network files
[ INFO ] Read network took 74.03 ms
[Step 5/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 1
[Step 6/11] Configuring input of the model
[ INFO ] Network input 'image_info' precision FP32, dimensions (NC): 1 3
[ INFO ] Network input 'image_tensor' precision U8, dimensions (NCHW): 1 3 800 1365
[ INFO ] Network output 'masks' precision FP32, dimensions (NCHW): 100 90 33 33
[ INFO ] Network output 'reshape_do_2d' precision FP32, dimensions (NC): 100 7
[Step 7/11] Loading the model to the device
[ ERROR ] Function contains several inputs and outputs with one friendly name!
Traceback (most recent call last):
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\main.py", line 255, in run
    exe_network = benchmark.load_network(ie_network)
  File "C:\Program Files (x86)\Intel\openvino_2021.4.582\python\python3.6\openvino\tools\benchmark\benchmark.py", line 63, in load_network
    num_requests=1 if self.api_type == 'sync' else self.nireq or 0)
  File "ie_api.pyx", line 367, in openvino.inference_engine.ie_api.IECore.load_network
  File "ie_api.pyx", line 379, in openvino.inference_engine.ie_api.IECore.load_network
RuntimeError: Function contains several inputs and outputs with one friendly name!

But the CPU and the GPU plugins work fine!

 

I attached also my mask rcnn model to this answer.

 

Regards,

Mouhammad

Zulkifli_Intel
Moderator
536 Views

Hello Mouhammad Abomoharam,

 

Thank you for sharing with us the model. On our side, we also see similar behavior when using the -d HETERO:GPU,CPU argument on mask_rcnn_resnet50_atrous_coco model.

 

We noticed you also open a new thread on GitHub #6645 for a similar issue. We will continue the investigation on GitHub. Therefore, we will close this thread and focus on GitHub. Are you okay with this arrangement?

 

Sincerely,

Zulkifli

MAbom
Beginner
533 Views

Hello Zulkifli,

 

Thanks for your reply.

 

Yes I think you can close this thread and focus on GitHub.

 

Thanks and regards,

Mouhammad

Zulkifli_Intel
Moderator
520 Views

Hello Mouhammad Abomoharam,

 

Thank you for your feedback. This thread will no longer be monitored since this issue will continue to be discussed on Github: https://github.com/openvinotoolkit/openvino/issues/6645

 

Sincerely,

Zulkifli


Reply