Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

No API found in Python for stateful models

Jia3Xu
Employee
705 Views

Hi

I've a model in pytorch and output it to onnx. Now I want to make it stateful.

Since my model does not contains LSTM/RNN/GRU cell so I use model optimizer to convert it by --transform "makestateful".  and generate IR.

I've checked IR which indeed has type="ReadValue" version="opset6" values. but the model's output and input disappears when pyopenvino compiled the model.

 

 

besides I also noticed in the document says "Working with these intermediate values of each iteration is enabled by special LowLatency and LowLatency2 transformations"

I my understand, the stateful models is supported only after transforms to LowLatency2, by using 

InferenceEngine::lowLatency2(cnnNetwork); 

but I haven't found any API in python can convert this. 

 

is there any way or examples of who python deal with stateful models?

0 Kudos
4 Replies
Peh_Intel
Moderator
675 Views

Hi Jia3Xu,


Thanks to reaching out to us.


Based on the documentation, it is only available for OpenVINO Runtime C++ API.


Let us get back to you at the earliest once we get further clarification on this.



Regards,

Peh


0 Kudos
Peh_Intel
Moderator
557 Views

Hi Jia3Xu,


You can use the following Python API to convert to LowLatency2.



from openvino.runtime import Model, Core

from openvino.runtime.passes import Manager, LowLatency2

 

core = Core()

model = core.read_model("/path/to/model")

manager = Manager()

manager.register_pass(LowLatency2())

manager.run_passes(model)

 

 

Regards,

Peh


0 Kudos
Hairul_Intel
Moderator
473 Views

Hi Jia3Xu,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

 

 

Regards,

Hairul


0 Kudos
Peh_Intel
Moderator
455 Views

Hi Jia3Xu,


Our developer already added the Network State introduction into GitHub page, which can be found here.


In addition, there will be Documentation for this stateful model API available in the future release.



Regards,

Peh


0 Kudos
Reply