Benchmark_app for Whisper model will failed on GPU

[WT_Jacko]

Hi Sirs,

Here is my BKC:

1. OS : ubuntu 22.04

2. Model : https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation

3. GPU : intel ARC pro 60

4. OpenVINO version : 2023.0

Description: 

Use OpenVINO 2023.0 Benchmark_app to run [openvino_notesbooks 227-whister] model will failed 

Error messages from GPU:

 From error messages that can know you need to provide dynamic input shape .  I have referred to the  227- notesbook, but no matter how I fill it out, I am unable to succeed. 

Thanks your help

BR

Jacko

[Aznie_Intel]

Hi Jacko,

Thanks for reaching out.

How did you run the benchmark_app with 227-whisper-subtitles-generation notebook? What command did you use? Please share your workaround and the error that you got for us to further investigate this.

Regards,

Aznie

[WT_Jacko]

Hi Aznie,

My Testing Steps are as below:

1.  Use openvino_notesbooks-227 to get whisper IR model Model 

=> https://github.com/openvinotoolkit/openvino_notebooks/tree/main/notebooks/227-whisper-subtitles-generation

2. The attachments are my Execution command and Error Message, please check it.

Thanks your help

BR

Jacko

[Aznie_Intel]

Hi Jacko,

Thanks for your patience.

The issue is expected since GPU only supports dynamic axis for batch size with a fixed upper limit. You may refer to this Dynamic Shapes documentation.

Regards,

Aznie

[WT_Jacko]

Hi Aznie,

Thanks for your response.

First , I don't understand what is your error messages. it is not same that I attached.

please see my another attachment that is call Execution command.png. 

Actually, I just wanna use benchmark_app to test whisper model  simply.  but i don't know how to run correct command include right parameters.

Thanks alot.

BR

Jacko

[Aznie_Intel]

Hi Jacko,

The error message shows that the 227-whisper-subtitles-generation notebook and model used are not supported on GPU plugin. The model can be inferred on CPU but not GPU.

There is nothing wrong with your command but I suspect that you will need to specify your input shapes of your model. Below is my benchmark_app inferred with whisper model on GPU plugin.

 Please share your model files .xml and .bin files for us to validate from our end.

Regards,

Aznie

[WT_Jacko]

Hi Aznie,

Here is my model link : https://drive.google.com/file/d/1wvkdvJTAqGh0yAT7cyPxQm7-ydsXbdXl/view?usp=drive_link

by the way,  in last message i cannot see your " benchmark_app "  command.

Thanks

BR

Jacko

[Aznie_Intel]

Hi Jacko,

I have requested access to your model files.

Regards,

Aznie

[WT_Jacko]

Hi Aznie,

Yes, please check it.

Thanks

BR

Jacko

[Aznie_Intel]

Hi Jacko,

From my validation, whisper_encoder (.onnx and .xml) are able to be run with Benchmark_app. Command used :benchmark_app -m C:\Users\Downloads\Whisper\Whisper\whisper_encoder.onnx -d GPU

However, the whisper_decoder(.onnx and .xml) both fail with data_shape error. Starting from the 2020.4 release, OpenVINO™ supports reading native ONNX models. You may also use your ONNX model without converting if nothing's wrong with it/supported.

Regards,

Aznie

[Aznie_Intel]

Hi Jacko,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question. 

Regards,

Aznie