Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6434 Discussions

Benchmark_app for Whisper model will failed on GPU

WT_Jacko
Beginner
2,154 Views

Hi Sirs,

 

Here is my BKC:

1. OS : ubuntu 22.04

2. Model : https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation

3. GPU : intel ARC pro 60

4. OpenVINO version : 2023.0

 

Description: 

Use OpenVINO 2023.0 Benchmark_app to run [openvino_notesbooks 227-whister] model will failed 

 

Error messages from GPU:

 From error messages that can know you need to provide dynamic input shape .  I have referred to the  227- notesbook, but no matter how I fill it out, I am unable to succeed. 

 

Thanks your help

BR

Jacko

0 Kudos
10 Replies
Aznie_Intel
Moderator
2,128 Views

Hi Jacko,

 

Thanks for reaching out.

 

How did you run the benchmark_app with 227-whisper-subtitles-generation notebook? What command did you use? Please share your workaround and the error that you got for us to further investigate this.

 

 

Regards,

Aznie

 

0 Kudos
WT_Jacko
Beginner
2,122 Views

Hi Aznie,

 

My Testing Steps are as below:

 

1.  Use openvino_notesbooks-227 to get whisper IR model Model 

=> https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation

2. The attachments are my Execution command and Error Message, please check it.

 

 

Thanks your help

BR

Jacko

0 Kudos
Aznie_Intel
Moderator
2,072 Views

Hi Jacko,

 

Thanks for your patience.

 

The issue is expected since GPU only supports dynamic axis for batch size with a fixed upper limit. You may refer to this Dynamic Shapes documentation.

expected gpu error.png

 

Regards,

Aznie

 

0 Kudos
WT_Jacko
Beginner
2,031 Views

Hi Aznie,

Thanks for your response.

First , I don't understand what is your error messages. it is not same that I attached.

please see my another attachment that is call Execution command.png. 

Actually, I just wanna use benchmark_app to test whisper model  simply.  but i don't know how to run correct command include right parameters.

 

 

Thanks alot.

BR

Jacko

0 Kudos
Aznie_Intel
Moderator
2,015 Views

Hi Jacko,

 

The error message shows that the 227-whisper-subtitles-generation notebook and model used are not supported on GPU plugin. The model can be inferred on CPU but not GPU.

 

There is nothing wrong with your command but I suspect that you will need to specify your input shapes of your model. Below is my benchmark_app inferred with whisper model on GPU plugin.

whisper (2).jpg

 

 Please share your model files .xml and .bin files for us to validate from our end.

 

 

Regards,

Aznie

 

0 Kudos
WT_Jacko
Beginner
2,010 Views

Hi Aznie,

 

Here is my model link : https://drive.google.com/file/d/1wvkdvJTAqGh0yAT7cyPxQm7-ydsXbdXl/view?usp=drive_link

 

by the way,  in last message i cannot see your " benchmark_app "  command.

 

Thanks

BR

Jacko

0 Kudos
Aznie_Intel
Moderator
1,989 Views

Hi Jacko,


I have requested access to your model files.



Regards,

Aznie


0 Kudos
WT_Jacko
Beginner
1,982 Views

Hi Aznie,

 

Yes, please check it.

 

Thanks

BR

Jacko

0 Kudos
Aznie_Intel
Moderator
1,960 Views

Hi Jacko,

 

From my validation, whisper_encoder (.onnx and .xml) are able to be run with Benchmark_app. Command used :benchmark_app -m C:\Users\Downloads\Whisper\Whisper\whisper_encoder.onnx -d GPU

whisper_encoder_jacko (2).jpg

However, the whisper_decoder(.onnx and .xml) both fail with data_shape error. Starting from the 2020.4 release, OpenVINO™ supports reading native ONNX models. You may also use your ONNX model without converting if nothing's wrong with it/supported.

 

 

Regards,

Aznie

 

0 Kudos
Aznie_Intel
Moderator
1,795 Views

Hi Jacko,


This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question. 



Regards,

Aznie


0 Kudos
Reply