Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6413 Discussions

Does OpenVINO support dynamic shape input/output model inference with iGPU or VPU?

WeiCheng_AcoustInTek
1,394 Views

I recently take over a project which use 2022.1 OpenVINO as its inference engine, and now I have a dynamic shape input/output IR model which is converted from onnx model (trained with PyTorch) by model optimizer .

This model can be inferenced with CPU(i7-11700k). However, I'm trying to inference this model with iGPU(UHD 750) and VPU (NCS2). but it would crash at compile_model() function. Then I found this document from OpenVINO's website:Inference Device Support , it said that only CPU support dynamic shape model inference. 

My question is: Is it ture that only CPU support dynamic shape input/output model inference?

Thanks in advance for any suggestion or insights.

0 Kudos
4 Replies
IntelSupport
Community Manager
1,335 Views

Hi WeiCheng_AcoustInTek,

 

Thanks for reaching out.

 

Yes, you are right. For the latest OpenVINO release (2022.3), the capability of dynamic shapes is only supported for CPU devices. Dynamic Shapes support on additional XPUs is expected in a future dot release.

 

You can always refer to the Release Notes for any latest updates.

 

 

Regards,

Aznie


0 Kudos
WeiCheng_AcoustInTek
1,265 Views

Hi, Aznie,
Thanks for you kindly replying.

Due to the portabilty of VPU,
I still highly interested in utiling VPU to inference Dynamic Shapes model.
I was wondering if OpenVINO develop team is trying to implement this feature on VPU? If so, would you like to provide a development process or roadmap that VPU supports for dynamic shape would be available?

Thanks in advance for your replying.

0 Kudos
IntelSupport
Community Manager
1,184 Views

Hi WeiCheng_Acoust,


We acknowledge your concern of continuing to use VPU for Dynamic shapes. Unfortunately, we don't have an update about when this will be available if planned for future releases. We strongly recommend referring to our release notes for the latest updates about the product.



Regards,

Aznie




0 Kudos
IntelSupport
Community Manager
1,088 Views

Hi WeiCheng_Acoust,

 

This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Aznie

 

 

0 Kudos
Reply