Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6502 Discussions

Keras | Tensorflow backend | Lstm Runtime Error

pankajrawat
Novice
4,032 Views

Below are the software version

 

 

Enviroment Details
=====================
openvino_fpga_2020.4.287  -- Intel cloud environment
Keras==2.4.3
tensorflow==2.2.0
onnx==1.7.0
onnxconverter-common==1.7.0
onnxruntime==1.3.0

 

 

 

Approach 1 - Keras to Onnx to Openvino [Backend tensorflow]

Keras model - running and giving results
Onnx model - running and giving results
Openvino   -  Model optimizer error while converting onnx to IR

 

 

[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.RNNSequenceNormalizeToIE.RNNSequenceNormalize'>): Something bad has happened with graph! Data node "lstm/HiddenStateResize" has 0 producers

 

 

Approach 2 - Keras to Openvino  [Backend tensorflow]

Model optimizer is giving below error while converting keras model to IR.

 

 

[ ERROR ]  Cannot infer shapes or values for node "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor".
[ ERROR ]  Tensorflow type 21 not convertible to numpy dtype.
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x7ff3db149ea0>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor" node. 
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38. 

 

 

 

Labels (1)
0 Kudos
8 Replies
Munesh_Intel
Moderator
4,010 Views

 

Hi Pankaj,

The errors that you are seeing seem to occur due to input shapes not being defined correctly or some unsupported TensorFlow operations.

General information regarding specifying input shapes is available here:

https://docs.openvinotoolkit.org/2020.4/openvino_docs_MO_DG_prepare_model_convert_model_Converting_Model_General.html#when_to_specify_input_shapes

 

TensorFlow Supported Operations are listed here:

https://docs.openvinotoolkit.org/2020.4/openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html#tensorflow_supported_operations

 

Please share more information about your model, the layers used if it's a custom model, command given to Model Optimizer to convert the trained model to Intermediate Representation (IR), and environment details (versions of OS, Python, CMake, etc.).

If possible, please share the trained model files for us to reproduce your issue.

 

Regards,

Munesh

 

0 Kudos
pankajrawat
Novice
3,976 Views
$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"

$ python  --version
Python 3.6.10

$ cmake --version
cmake version 3.10.2

 

Command Used to convert Onnx to IR

(cenv) u47404@s099-n003:~/intelmac$ python /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --input_model models/keras/timeseries_enode.onnx --input_shape [128,50,2]
Model Optimizer arguments:
Common parameters:
        - Path to the Input Model:      /home/u47404/intelmac/models/keras/timeseries_enode.onnx
        - Path for generated IR:        /home/u47404/intelmac/.
        - IR output name:       timeseries_enode
        - Log level:    ERROR
        - Batch:        Not specified, inherited from the model
        - Input layers:         Not specified, inherited from the model
        - Output layers:        Not specified, inherited from the model
        - Input shapes:         [128,50,2]
        - Mean values:  Not specified
        - Scale values:         Not specified
        - Scale factor:         Not specified
        - Precision of IR:      FP32
        - Enable fusing:        True
        - Enable grouped convolutions fusing:   True
        - Move mean values to preprocess section:       False
        - Reverse input channels:       False
ONNX specific parameters:
Model Optimizer version: 
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.RNNSequenceNormalizeToIE.RNNSequenceNormalize'>): Something bad has happened with graph! Data node "lstm/HiddenStateResize" has 0 producers

 

Command Used to convert from Keras to IR

python /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --saved_model_dir models/keras/timeseries_enode.keras_tf/  --input_shape [128,50,2]

 

Sample Model - Layers i am using

obj = Sequential()
obj.add(LSTM(units=128, return_sequences=False, input_shape=(50, 2)))
obj.add(RepeatVector(20))
obj.add(LSTM(units=128, return_sequences=True))
obj.add(Dense(units=128))
obj.add(TimeDistributed(Dense(units=2)))
optimizer = Adam(0.001)
0 Kudos
Munesh_Intel
Moderator
3,926 Views

Hi Pankaj,

“TensorArrayUnstack” is not a supported TensorFlow operation in OpenVINO. The TensorFlow supported operations are available at the following page: https://docs.openvinotoolkit.org/2020.4/openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html#tensorflow_supported_operations


Regards,

Munesh


0 Kudos
pankajrawat
Novice
3,913 Views

Based on the error. It seems like the above mentioned tensor operation is part of LSTM Layer itself, do you know a working example of the same.

I Replaced below but same error is coming. 

Note: This syntax is of tensorflow 2.0 and not keras, as the error is same for both tensorflow and keras with tensorflow as backend

 

lstm = tf.keras.layers.LSTM(hidden_nodes, return_sequences=True)(rv)

with

lstm = tf.keras.layers.RNN(tf.keras.layers.LSTMCell(hidden_nodes), return_sequences=True)(rv)

 

 

 

[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/functional_1/lstm/PartitionedCall/TensorArrayUnstack/TensorListFromTensor" node. 

 

 

0 Kudos
Munesh_Intel
Moderator
3,904 Views

Hi Pankaj,

TensorFlow2 support has only recently been implemented in OpenVINO and support for LSTM is not validated yet. Having said that, I would suggest you have a look at the following two methods:


(1)  Configure the LSTM layer to both return sequences and return states, as follows:


lstm = tf.keras.layers.LSTM(1, return_sequences=True, return_state=True)


Related information is available at the following page:

https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM


(2)  The following community thread provides a workaround, but this is for TensorFlow 1.

https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/cannot-convert-LSTM-keras-model-to-IR-files/td-p/1138027


Regards,

Munesh


0 Kudos
pankajrawat
Novice
3,872 Views

As per suggestion i modified the code to add return state. After adding sate=True, the lstm layer was returning 3 output.

[<tf.Tensor 'lstm_1/PartitionedCall:1' shape=(None, 20, 128) dtype=float32>, <tf.Tensor 'lstm_1/PartitionedCall:2' shape=(None, 128) dtype=float32>, <tf.Tensor 'lstm_1/PartitionedCall:3' shape=(None, 128) dtype=float32>]

For our usecase we need complete output from 2nd LSTM  => (None, 20, 128).

The Expected one was the  0th Index One. So modified code looks like.

    input_x = tf.keras.Input(shape=(COL_X, FEATURES), name='input_layer')
    
    lstm = tf.keras.layers.LSTM(hidden_nodes)(input_x)
    rv = tf.keras.layers.RepeatVector(COL_Y)(lstm)

    #lstm = tf.keras.layers.LSTM(hidden_nodes, return_sequences=True)(rv)
    lstm = tf.keras.layers.LSTM(hidden_nodes, 
                       return_sequences=True, 
                       return_state=True)(rv)
    #lstm = tf.keras.layers.RNN(tf.keras.layers.LSTMCell(hidden_nodes), return_sequences=True)(rv)
    print(lstm)
    dense = tf.keras.layers.Dense(128)(lstm[0])
    dense = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(FEATURES))(dense)
    
    model = tf.keras.Model(inputs=input_x, outputs=dense)
    optimizer = tf.keras.optimizers.Adam(lr)
    model.compile(optimizer=optimizer, loss='mse', metrics=['mse'])

 

The code compiles and gives output in tensorflow, but when converted to openvino same error is coming.

[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/functional_1/lstm/PartitionedCall/TensorArrayUnstack/TensorListFromTensor" node. 
 For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38. 

 

0 Kudos
Munesh_Intel
Moderator
3,841 Views

Hi Pankaj,


OpenVINO™ support for TensorFlow 2 models, which has only been recently implemented,  is in preview (aka Beta), which means limited and not of production quality yet. Also, OpenVINO™ does not support models with Keras RNN and Embedding layers.


These information is provided at the following page:

https://docs.openvinotoolkit.org/2020.4/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#Convert_From_TF2X


Regards,

Munesh


0 Kudos
Munesh_Intel
Moderator
3,826 Views

Hi Pankaj,


This thread will no longer be monitored since we have provided references. If you need any additional information from Intel, please submit a new question. 


Regards,

Munesh


0 Kudos
Reply