Can not convert SSD to IR (KEras and tensorflow)

[LAI__PEI_YU]

Hello.

I am new to openvino and using Openvino 2020.1.023

I use the SSD300 from the link below:

https://github.com/balancap/SSD-Tensorflow

I freeze the model using the code below

with tf.Session(graph=tf.Graph()) as sess:
        # We import the meta graph in the current default Graph
        saver = tf.train.import_meta_graph(input_checkpoint + '.meta', clear_devices=clear_devices)
        graph_def = tf.get_default_graph().as_graph_def()
        node_list=[n.name for n in graph_def.node]
        # We restore the weights
        saver.restore(sess, input_checkpoint)
        # Dddd=output_node_names.split(",")

        # We use a built-in TF helper to export variables to constants
        output_graph_def = tf.graph_util.convert_variables_to_constants(
            sess, # The session is used to retrieve the weights
            tf.get_default_graph().as_graph_def(), # The graph_def is used to retrieve the nodes 
            node_list # The output node names are used to select the usefull nodes
        ) 

        # tf.train.write_graph(sess.graph_def, '.', 'tfdroid.pbtxt')

        # Finally we serialize and dump the output graph to the filesystem
        with tf.gfile.GFile(output_graph, "wb") as f:
            f.write(output_graph_def.SerializeToString())
        print("%d ops in the final graph." % len(output_graph_def.node))

    return output_graph_def

and then use the command line:

python3 mo_tf.py --input_model frozen_model.pb --output_dir /home/ibox/Desktop/  --input_shape [1,300,300,3]  --log_level=DEBUG

the debug text is in the attached file(TensorFlow_SSD300(DEBUG))

I have also tried keras ssd 300 from the link below:

https://github.com/pierluigiferrari/ssd_keras

and convert to .pb used the code below.

def export_keras_to_tf(input_model, output_model, num_output):
    print('Loading Keras model: ', input_model)

    keras_model = load_model(input_model,custom_objects={'loss': bce_jaccard_loss,'dice_coeff': dice_coeff})

    print(keras_model.summary())

    predictions = [None] * num_output
    predrediction_node_names = [None] * num_output

    for i in range(num_output):
        predrediction_node_names = 'output_node' + str(i)
        predictions = tf.identity(keras_model.outputs, name=predrediction_node_names)

    sess = K.get_session()

    constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), predrediction_node_names)
    infer_graph = graph_util.remove_training_nodes(constant_graph) 

    graph_io.write_graph(infer_graph, '.', output_model, as_text=False)

and use the command line:

python3 mo.py --input_model ddd.h5.pb  --output_dir /home/ibox/Desktop/ --input_shape [1,300,300,3] --log_level=DEBUG

The debug text is in the attached file(Keras_SSD300). I have check the output shape in debug file, it is the same as what i have got using keras model.summary() method.

Really don't know is going on, really appreciate your help!

[Max_L_Intel]

Hello, LAI, PEI YU.

Please carefully check this documentation on how to convert a frozen TensorFlow model - https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html#Convert_From_TF

You might need to specify the following parameters:

--tensorflow_use_custom_operations_config  
​​​​​​​--tensorflow_object_detection_api_pipeline_config

Also, in the following thread, a user converted a SSD mobilenet to IR - https://software.intel.com/en-us/forums/computer-vision/topic/805387
This might be similar to your case.

Thanks.

[LAI__PEI_YU]

Dear Max

Thanks for your reply,

I'm not using the object detection api model, so is it ok with me to use the config file from the object detection api model?

[Max_L_Intel]

Hello LAI, PEI YU

Do you use any other API and/or sample application from OpenVINO toolkit distribution? As I'm aware of, SSD models are mostly utilized for object detection purposes.

If your case is similar to this one, please give it a try with the object detection config file to see if it works.

Best regards, Max.

[LAI__PEI_YU]

Dear Max,

Thanks for your reply,

I have added the parameter you suggested, but it doesn't work.

python3 mo_tf.py --input_model /home/ibox/Desktop/BusParkingDetection/frozen_model/frozen_model.pb --output_dir /home/ibox/Desktop/BusParkingDetection/ --input_shape [1,300,300,3]  --tensorflow_use_custom_operations_config  /opt/intel/openvino_2020.1.023/deployment_tools/model_optimizer/extensions/front/tf/ssd_support.json  --tensorflow_object_detection_api_pipeline_config /home/ibox/Downloads/ssd_mobilenet_v1_coco_2018_01_28/pipeline.config

Maybe I should create my own config file? Do you have any suggestion?

Really appreciate your help!

[Max_L_Intel]

Hi, LAI, PEI YU.

SSD-300 model that you are using is based on Object Detection API.
For converting the TensorFlow version of this model please try to use one of the following .json and .config file pairs, according to different conditions:

--transformations_config <path_to_subgraph_replacement_configuration_file.json> — A subgraph replacement configuration file with transformations description. For the models downloaded from the TensorFlow* Object Detection API zoo, you can find the configuration files in the <INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf directory. Use:

ssd_v2_support.json — for frozen SSD topologies from the models zoo version up to 1.13.X inclusively 
ssd_support_api_v.1.14.json (I suppose this one would be more appropriate for your case) — for frozen SSD topologies trained manually using the TensorFlow* Object Detection API version 1.14 or higher

--tensorflow_object_detection_api_pipeline_config <path_to_pipeline.config> — A special configuration file that describes the topology hyper-parameters and structure of the TensorFlow Object Detection API model. For the models downloaded from the TensorFlow* Object Detection API zoo, the configuration file is named pipeline.config. If you plan to train a model yourself, you can find templates for these files in the models repository - https://github.com/tensorflow/models/tree/master/research/object_detection/samples/configs 

Also, you could have a chance to try this officially validated Caffe-version of SSD-300 model for OpenVINO toolkit - https://github.com/opencv/open_model_zoo/tree/master/models/public/ssd300

Best regards, Max.

[LAI__PEI_YU]

Dear Max,

Thanks for your reply,

I have added what you suggested, but it didn't work.My ssd model is vgg based, I think it is different from the models in detection model zoo. The error is still the same:

Number of elements in input [ 1 300 300 300 3] and output [300, 300, 3] of reshape node ssd_preprocessing_train/resize_image/Reshape mismatch

I know tensorflow model does not have well-define shapes, so I used keras as well. I have checked  the input/output  shape, still can not find the problem.

the error is :  Can't permute attrs for node pool4/MaxPool, the same as link below:

https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit/topic/844525

So maybe I should wait for intel to fix this? or you have other suggestions  ?

Because I don't have an device for training openvino model (lack of intel gpu), so I think converting it from my PC(not intel gpu) is a more appropriate way.

Thanks for your help!

[Max_L_Intel]

Hi LAI, PEI YU.

With regards to TensorFlow model, I just recently converted VGG-16 based model. Please see my reply in this topic - https://software.intel.com/en-us/forums/intel-distribution-of-openvino-toolkit/topic/848743

If you face "AssertionError: Bad argument number", you might need to use 

pip3 install gast==0.2.2

On Keras model, yes, it has been already reported to developers team.