- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Below are the software version
Enviroment Details
=====================
openvino_fpga_2020.4.287 -- Intel cloud environment
Keras==2.4.3
tensorflow==2.2.0
onnx==1.7.0
onnxconverter-common==1.7.0
onnxruntime==1.3.0
Approach 1 - Keras to Onnx to Openvino [Backend tensorflow]
Keras model - running and giving results
Onnx model - running and giving results
Openvino - Model optimizer error while converting onnx to IR
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.RNNSequenceNormalizeToIE.RNNSequenceNormalize'>): Something bad has happened with graph! Data node "lstm/HiddenStateResize" has 0 producers
Approach 2 - Keras to Openvino [Backend tensorflow]
Model optimizer is giving below error while converting keras model to IR.
[ ERROR ] Cannot infer shapes or values for node "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor".
[ ERROR ] Tensorflow type 21 not convertible to numpy dtype.
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function tf_native_tf_node_infer at 0x7ff3db149ea0>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/sequential/lstm/StatefulPartitionedCall/TensorArrayUnstack/TensorListFromTensor" node.
For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Pankaj,
The errors that you are seeing seem to occur due to input shapes not being defined correctly or some unsupported TensorFlow operations.
General information regarding specifying input shapes is available here:
TensorFlow Supported Operations are listed here:
Please share more information about your model, the layers used if it's a custom model, command given to Model Optimizer to convert the trained model to Intermediate Representation (IR), and environment details (versions of OS, Python, CMake, etc.).
If possible, please share the trained model files for us to reproduce your issue.
Regards,
Munesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.2 LTS"
$ python --version
Python 3.6.10
$ cmake --version
cmake version 3.10.2
Command Used to convert Onnx to IR
(cenv) u47404@s099-n003:~/intelmac$ python /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --input_model models/keras/timeseries_enode.onnx --input_shape [128,50,2]
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: /home/u47404/intelmac/models/keras/timeseries_enode.onnx
- Path for generated IR: /home/u47404/intelmac/.
- IR output name: timeseries_enode
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: [128,50,2]
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: False
- Reverse input channels: False
ONNX specific parameters:
Model Optimizer version:
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.RNNSequenceNormalizeToIE.RNNSequenceNormalize'>): Something bad has happened with graph! Data node "lstm/HiddenStateResize" has 0 producers
Command Used to convert from Keras to IR
python /opt/intel/openvino/deployment_tools/model_optimizer/mo.py --saved_model_dir models/keras/timeseries_enode.keras_tf/ --input_shape [128,50,2]
Sample Model - Layers i am using
obj = Sequential()
obj.add(LSTM(units=128, return_sequences=False, input_shape=(50, 2)))
obj.add(RepeatVector(20))
obj.add(LSTM(units=128, return_sequences=True))
obj.add(Dense(units=128))
obj.add(TimeDistributed(Dense(units=2)))
optimizer = Adam(0.001)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Pankaj,
“TensorArrayUnstack” is not a supported TensorFlow operation in OpenVINO. The TensorFlow supported operations are available at the following page: https://docs.openvinotoolkit.org/2020.4/openvino_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html#tensorflow_supported_operations
Regards,
Munesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Based on the error. It seems like the above mentioned tensor operation is part of LSTM Layer itself, do you know a working example of the same.
I Replaced below but same error is coming.
Note: This syntax is of tensorflow 2.0 and not keras, as the error is same for both tensorflow and keras with tensorflow as backend
lstm = tf.keras.layers.LSTM(hidden_nodes, return_sequences=True)(rv)
with
lstm = tf.keras.layers.RNN(tf.keras.layers.LSTMCell(hidden_nodes), return_sequences=True)(rv)
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/functional_1/lstm/PartitionedCall/TensorArrayUnstack/TensorListFromTensor" node.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Pankaj,
TensorFlow2 support has only recently been implemented in OpenVINO and support for LSTM is not validated yet. Having said that, I would suggest you have a look at the following two methods:
(1) Configure the LSTM layer to both return sequences and return states, as follows:
lstm = tf.keras.layers.LSTM(1, return_sequences=True, return_state=True)
Related information is available at the following page:
https://www.tensorflow.org/api_docs/python/tf/keras/layers/LSTM
(2) The following community thread provides a workaround, but this is for TensorFlow 1.
Regards,
Munesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
As per suggestion i modified the code to add return state. After adding sate=True, the lstm layer was returning 3 output.
[<tf.Tensor 'lstm_1/PartitionedCall:1' shape=(None, 20, 128) dtype=float32>, <tf.Tensor 'lstm_1/PartitionedCall:2' shape=(None, 128) dtype=float32>, <tf.Tensor 'lstm_1/PartitionedCall:3' shape=(None, 128) dtype=float32>]
For our usecase we need complete output from 2nd LSTM => (None, 20, 128).
The Expected one was the 0th Index One. So modified code looks like.
input_x = tf.keras.Input(shape=(COL_X, FEATURES), name='input_layer')
lstm = tf.keras.layers.LSTM(hidden_nodes)(input_x)
rv = tf.keras.layers.RepeatVector(COL_Y)(lstm)
#lstm = tf.keras.layers.LSTM(hidden_nodes, return_sequences=True)(rv)
lstm = tf.keras.layers.LSTM(hidden_nodes,
return_sequences=True,
return_state=True)(rv)
#lstm = tf.keras.layers.RNN(tf.keras.layers.LSTMCell(hidden_nodes), return_sequences=True)(rv)
print(lstm)
dense = tf.keras.layers.Dense(128)(lstm[0])
dense = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(FEATURES))(dense)
model = tf.keras.Model(inputs=input_x, outputs=dense)
optimizer = tf.keras.optimizers.Adam(lr)
model.compile(optimizer=optimizer, loss='mse', metrics=['mse'])
The code compiles and gives output in tensorflow, but when converted to openvino same error is coming.
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "StatefulPartitionedCall/functional_1/lstm/PartitionedCall/TensorArrayUnstack/TensorListFromTensor" node.
For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Pankaj,
OpenVINO™ support for TensorFlow 2 models, which has only been recently implemented, is in preview (aka Beta), which means limited and not of production quality yet. Also, OpenVINO™ does not support models with Keras RNN and Embedding layers.
These information is provided at the following page:
Regards,
Munesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Pankaj,
This thread will no longer be monitored since we have provided references. If you need any additional information from Intel, please submit a new question.
Regards,
Munesh
![](/skins/images/D2683F18326913BBA0436CB7114DD569/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page