Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6434 Discussions

[OpenVINO 2023.0] Benchmark_app for ONNX Model on GPU will failed, but can run on CPU properly.

WT_Jacko
Beginner
615 Views

Hi Sirs,

 

Here is my BKC:

1. OS : ubuntu 22.04

2. Model : My Onnx model : https://www.dropbox.com/s/l8d7qorwv37r04s/bbb3.zip?dl=0 

3. GPU : intel ARC pro 60

4. OpenVINO version : 2023.0

 

Description: 

Use OpenVINO 2023.0 Benchmark_app to run my onnx model, GPU will show error message. But CPU can run properly.

 

Error messages from GPU:

Screenshot from 2023-06-27 17-44-46.png

 From error messages that can only know cause by dynamic shape . Hope can get more detail information. If possible, please let me know if this model can be executed on a GPU.

 

Thanks your help.

BR

Jacko

 

 

0 Kudos
3 Replies
Iffa_Intel
Moderator
587 Views

Hi,

 

I did a quick check & I could observe the same thing.

The model can be inferred on CPU but not GPU.

dynamic.PNG

 

The reason is, currently, dynamic shape support for GPU is a preview feature and has the following limitations:

  • It mainly supports NLP models (Natural Language Processing). Not all operations and optimization passes support dynamic shapes. As a result, a given model may crash or experience significant performance drops.
  • Due to the dominant runtime overhead on the host device, dynamic shapes may perform worse than static shapes on a discrete GPU.
  • Dynamic rank is not supported.

 

You may refer here for further info.

 

Cordially,

Iffa

 

0 Kudos
WT_Jacko
Beginner
574 Views

Hi Iffa,

 

Thanks your response.

 

BR

Jacko

0 Kudos
Iffa_Intel
Moderator
543 Views

Hi,


Glad that helps!.


Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 



Cordially,

Iffa


0 Kudos
Reply