Does OpenVINO support dynamic shape input/output model inference with iGPU or VPU?

[WeiCheng_AcoustInTek]

I recently take over a project which use 2022.1 OpenVINO as its inference engine, and now I have a dynamic shape input/output IR model which is converted from onnx model (trained with PyTorch) by model optimizer .

This model can be inferenced with CPU(i7-11700k). However, I'm trying to inference this model with iGPU(UHD 750) and VPU (NCS2). but it would crash at compile_model() function. Then I found this document from OpenVINO's website:Inference Device Support , it said that only CPU support dynamic shape model inference. 

My question is: Is it ture that only CPU support dynamic shape input/output model inference?

Thanks in advance for any suggestion or insights.

[IntelSupport]

Hi WeiCheng_AcoustInTek,

Thanks for reaching out.

Yes, you are right. For the latest OpenVINO release (2022.3), the capability of dynamic shapes is only supported for CPU devices. Dynamic Shapes support on additional XPUs is expected in a future dot release.

You can always refer to the Release Notes for any latest updates.

Regards,

Aznie

[WeiCheng_AcoustInTek]

Hi, Aznie,
Thanks for you kindly replying.

Due to the portabilty of VPU,
I still highly interested in utiling VPU to inference Dynamic Shapes model.
I was wondering if OpenVINO develop team is trying to implement this feature on VPU? If so, would you like to provide a development process or roadmap that VPU supports for dynamic shape would be available?

Thanks in advance for your replying.

[IntelSupport]

Hi WeiCheng_Acoust,

We acknowledge your concern of continuing to use VPU for Dynamic shapes. Unfortunately, we don't have an update about when this will be available if planned for future releases. We strongly recommend referring to our release notes for the latest updates about the product.

Regards,

Aznie

[IntelSupport]

Hi WeiCheng_Acoust,

This thread will no longer be monitored since we have provided information. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie