No API found in Python for stateful models

[Jia3Xu]

Hi

I've a model in pytorch and output it to onnx. Now I want to make it stateful.

Since my model does not contains LSTM/RNN/GRU cell so I use model optimizer to convert it by --transform "makestateful".  and generate IR.

I've checked IR which indeed has type="ReadValue" version="opset6" values. but the model's output and input disappears when pyopenvino compiled the model.

besides I also noticed in the document says "Working with these intermediate values of each iteration is enabled by special LowLatency and LowLatency2 transformations"

I my understand, the stateful models is supported only after transforms to LowLatency2, by using 

InferenceEngine::lowLatency2(cnnNetwork); 

but I haven't found any API in python can convert this. 

is there any way or examples of who python deal with stateful models?

[Peh_Intel]

Hi Jia3Xu,

Thanks to reaching out to us.

Based on the documentation, it is only available for OpenVINO Runtime C++ API.

Let us get back to you at the earliest once we get further clarification on this.

Regards,

Peh

[Peh_Intel]

Hi Jia3Xu,

You can use the following Python API to convert to LowLatency2.

from openvino.runtime import Model, Core

from openvino.runtime.passes import Manager, LowLatency2

core = Core()

model = core.read_model("/path/to/model")

manager = Manager()

manager.register_pass(LowLatency2())

manager.run_passes(model)

Regards,

Peh

[Hairul_Intel]

Hi Jia3Xu,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Regards,

Hairul

[Peh_Intel]

Hi Jia3Xu,

Our developer already added the Network State introduction into GitHub page, which can be found here.

In addition, there will be Documentation for this stateful model API available in the future release.

Regards,

Peh