Cannot convert ConvLSTM2D layer to IR

[Yadav__Deepanshu]

I have a model of the form given below

class Conv3DModel(tf.keras.Model):
    def __init__(self):
        super(Conv3DModel, self).__init__()
        # Convolutions
        self.conv1 = tf.compat.v2.keras.layers.Conv3D(32, (3, 3, 3), activation='relu', name="conv1", data_format='channels_last')
        self.pool1 = tf.keras.layers.MaxPool3D(pool_size=(2, 2, 2), data_format='channels_last')
        self.conv2 = tf.compat.v2.keras.layers.Conv3D(64, (3, 3, 3), activation='relu', name="conv1", data_format='channels_last')
        self.pool2 = tf.keras.layers.MaxPool3D(pool_size=(2, 2,2), data_format='channels_last')

        # LSTM & Flatten
        self.convLSTM =tf.keras.layers.ConvLSTM2D(40, (3, 3))
        self.flatten =  tf.keras.layers.Flatten(name="flatten")

        # Dense layers
        self.d1 = tf.keras.layers.Dense(128, activation='relu', name="d1")
        self.out = tf.keras.layers.Dense(6, activation='softmax', name="output")
    

    def call(self, x):
        x = self.conv1(x)
        x = self.pool1(x)
        x = self.conv2(x)
        x = self.pool2(x)
        x = self.convLSTM(x)
        x = self.flatten(x)
        x = self.d1(x)
        return self.out(x)

I tried to convert the model into IR. The model is here .

I have trained this mode in tensorflow 1.15.

Now I tried to run the command

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --saved_model_dir jester_trained_models/3dcnn-basic/  --output_dir /home/deepanshu/open_vino/udacity_project_custom_model/

Now i got the following error

Model Optimizer arguments:

Common parameters: -

Path to the Input Model: None

- Path for generated IR: /home/deepanshu/open_vino/udacity_project_custom_model/

- IR output name: saved_model

- Log level: ERROR

- Batch: Not specified, inherited from the model

- Input layers: Not specified, inherited from the model

- Output layers: Not specified, inherited from the model

- Input shapes: Not specified, inherited from the model

- Mean values: Not specified

- Scale values: Not specified

- Scale factor: Not specified

- Precision of IR: FP32

- Enable fusing: True

- Enable grouped convolutions fusing: True

- Move mean values to preprocess section: False

- Reverse input channels: False

TensorFlow specific parameters:

- Input model in text protobuf format: False

- Path to model dump for TensorBoard: None

- List of shared libraries with TensorFlow custom layers implementation: None

- Update the configuration file with input/output node names: None

- Use configuration file used to generate the model with Object Detection API: None

- Operations to offload: None

- Patterns to offload: None

- Use the config file: None

Model Optimizer version: 2020.1.0-61-gd349c3ba4a

[ ERROR ] Unexpected exception happened during extracting attributes for node conv3d_model/conv_lst_m2d/bias/Read/ReadVariableOp. Original exception message: 'ascii' codec can't decode byte 0xc9 in position 1: ordinal not in range(128)

As far as  I can see it is the tf.keras.layers.ConvLSTM2D(40, (3, 3)) causing problems . I am kind of stuck here . Can anyone tell me where can I proceed further ?

Thanks

Deepanshu

[Yadav__Deepanshu]

Now I rejected the above tensorflow implementation and used keras . My h5 model developed was converted into .pb format using this post.

Now I ran the model optimizer on this .pb file. Using the command 

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model /home/deepanshu/ml_playground/jester_freezed/tf_model.pb  --output_dir /home/deepanshu/open_vino/udacity_project_custom_model/  --input_shape=[1,30,64,64,1] --data_type FP32

Now i am facing another issue . The issue here is point no. 97 on this post.

So my model contains a cycle and model optimizer does not know a way to convert it. Has anybody faced this issue before ?

Please help.

Here is the model .

Here is the defination of the model in keras

from keras.models import Sequential

from keras.layers import Conv3D , MaxPool3D,Flatten ,Dense

from keras.layers.convolutional_recurrent import ConvLSTM2D

import keras

model = Sequential()

model.add(Conv3D(32, (3, 3, 3), 

         name="conv1" , input_shape=(30, 64, 64,1) ,  data_format='channels_last',

        activation='relu') )

model.add(MaxPool3D(pool_size=(2, 2, 2), data_format='channels_last'))

model.add(Conv3D(64, (3, 3, 3), activation='relu', name="conv2", data_format='channels_last'))

model.add(MaxPool3D(pool_size=(2, 2,2), data_format='channels_last'))

model.add(ConvLSTM2D(40, (3, 3)))

model.add(Flatten(name="flatten"))

model.add(Dense(128, activation='relu', name="d1"))

model.add(Dense(6, activation='softmax', name="output"))

[Yadav__Deepanshu]

Actually the script to convert from h5 to .pb was not good enough. Use this .

And now convert your model to IR using

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model ml_playground/try_directory/tf_model.pb   --output_dir /home/deepanshu/open_vino/udacity_project_custom_model/  --input_shape=[1,30,64,64,1] --data_type FP32

Now the IR files have been generated . Now once again I ran into problem. Please help this error is beyond my current level.

This is how it is produced

 

from openvino.inference_engine import IENetwork, IECore

def load_to_IE(model_xml, cpu_extension=None):
    # Load the Inference Engine API
    plugin = IECore()

    # Load IR files into their related class
    model_bin = os.path.splitext(model_xml)[0] + ".bin"
    net = IENetwork(model=model_xml, weights=model_bin)

# call this function using 

load_to_IE('model/tf_model.xml')

The generated files (IR) are here uploaded in the attachement.