DL Workbench Issue: Cannot convert RGBD SSD model to OpenVINO IR format

[SaifVazir]

Hi, I've trained a 4 channel (RGB+D) ssdlite model using Tensorflow Object Detection API(v1.5). Unfortunately, I haven't been able to convert this model to OpenVINO IR format using DL Workbench. 

```

Stopped shape/value propagation at "Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/range_2" node
[ WARNING ] Use of deprecated cli option --tensorflow_use_custom_operations_config detected. Option use in the following releases will be fatal. Please use --transformations_config cli option instead
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: /home/workbench/.workbench/models/3/original/mbnetv2_depth.pb
- Path for generated IR: /home/workbench/.workbench/models/4/original
- IR output name: mbnetv2_depth
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: image_tensor
- Output layers: Not specified, inherited from the model
- Input shapes: [1,300,300,4]
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP16
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: True
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: /home/workbench/.workbench/models/3/original/pipeline.config
- Use the config file: /opt/intel/openvino_2021.2.185/deployment_tools/model_optimizer/extensions/front/tf/ssd_support_api_v1.14.json
Model Optimizer version: 2021.2.0-1877-176bdf51370-releases/2021/2
Progress: [ ] 0.31% done
Progress: [ ] 0.62% done
Progress: [ ] 0.94% done
Progress: [ ] 1.25% done
Progress: [ ] 1.56% done
Progress: [ ] 1.88% done
Progress: [ ] 2.19% done
Progress: [ ] 2.50% done
Progress: [ ] 2.81% done
Progress: [ ] 3.12% done
Progress: [ ] 3.44% done
Progress: [ ] 3.75% done
Progress: [ ] 4.06% done
Progress: [ ] 4.38% done
Progress: [ ] 4.69% done
Progress: [. ] 5.00% done
Progress: [. ] 5.31% done
Progress: [. ] 5.62% done
Progress: [. ] 5.94% done
Progress: [. ] 6.25% done
Progress: [. ] 6.56% done
Progress: [. ] 6.88% done
Progress: [. ] 7.19% done
Progress: [. ] 7.50% done
Progress: [. ] 7.81% done
Progress: [. ] 8.12% done
Progress: [. ] 8.44% done
Progress: [. ] 8.75% done
Progress: [. ] 9.06% done
Progress: [. ] 9.38% done
Progress: [. ] 9.69% done
Progress: [.. ] 10.00% done
Progress: [.. ] 10.31% done
Progress: [.. ] 10.62% done
Progress: [.. ] 10.94% done
Progress: [.. ] 11.25% done
Progress: [.. ] 11.56% done
Progress: [.. ] 11.88% done
Progress: [.. ] 12.19% done
Progress: [.. ] 12.50% done
Progress: [.. ] 12.81% done
Progress: [.. ] 13.12% done
Progress: [.. ] 13.44% done
Progress: [.. ] 13.75% done
Progress: [.. ] 14.06% done
Progress: [.. ] 14.37% done
Progress: [.. ] 14.69% done
Progress: [... ] 15.00% done
Progress: [... ] 15.31% done
Progress: [... ] 15.62% done
Progress: [... ] 15.94% done
Progress: [... ] 16.25% done
Progress: [... ] 16.56% done
Progress: [... ] 16.88% done
Failed to match nodes from custom replacement description with id 'ObjectDetectionAPIPreprocessorReplacement':
It means model and custom replacement description are incompatible.
Try to correct custom replacement description according to documentation with respect to model node names
Progress: [... ] 17.19% done
Progress: [... ] 17.50% done
Progress: [... ] 17.81% done
Progress: [... ] 18.12% done
Progress: [... ] 18.44% done
Progress: [... ] 18.75% done
Progress: [... ] 19.06% done
Progress: [... ] 19.38% done
Progress: [... ] 19.69% done
Progress: [.... ] 20.00% done
Progress: [.... ] 20.31% done
Progress: [.... ] 20.62% done
Progress: [.... ] 20.94% done
Progress: [.... ] 21.25% done
Progress: [.... ] 21.56% done
Progress: [.... ] 21.88% done
Progress: [.... ] 22.19% done
Progress: [.... ] 22.50% done
Progress: [.... ] 22.81% done
Progress: [.... ] 23.12% done
Progress: [.... ] 23.44% done
Progress: [.... ] 23.75% done
Progress: [.... ] 24.06% done
Progress: [.... ] 24.38% done
Progress: [.... ] 24.69% done
Progress: [..... ] 25.00% done
Failed to match nodes from custom replacement description with id 'ObjectDetectionAPISSDPostprocessorReplacement':
It means model and custom replacement description are incompatible.
Try to correct custom replacement description according to documentation with respect to model node names
Progress: [..... ] 25.31% done
Progress: [..... ] 25.62% done
Progress: [..... ] 25.94% done
Progress: [..... ] 26.25% done
Progress: [..... ] 26.56% done
Progress: [..... ] 26.88% done
Progress: [..... ] 27.19% done
Progress: [..... ] 27.50% done
Progress: [..... ] 27.81% done
Progress: [..... ] 28.12% done
Progress: [..... ] 28.44% done
Progress: [..... ] 28.75% done
Progress: [..... ] 29.06% done
Progress: [..... ] 29.38% done
Progress: [..... ] 29.69% done
Progress: [...... ] 30.00% done
Progress: [...... ] 30.31% done
Progress: [...... ] 30.63% done
Progress: [...... ] 30.94% done
Progress: [...... ] 31.25% done
Progress: [...... ] 31.56% done
Progress: [...... ] 31.87% done
Progress: [...... ] 32.19% done
Progress: [...... ] 32.50% done
Progress: [...... ] 32.81% done
Progress: [...... ] 33.12% done
Progress: [...... ] 33.44% done
Progress: [...... ] 33.75% done
Progress: [...... ] 34.06% done
Progress: [...... ] 34.38% done
Progress: [...... ] 34.69% done
Progress: [....... ] 35.00% done
Progress: [....... ] 35.31% done
Progress: [....... ] 35.62% done
Progress: [....... ] 35.94% done
Progress: [....... ] 36.25% done
Progress: [....... ] 36.56% done
Progress: [....... ] 36.88% done
Progress: [....... ] 37.19% done
Progress: [....... ] 37.50% done
Progress: [....... ] 37.81% done
Progress: [....... ] 38.12% done
Progress: [....... ] 38.44% done
Progress: [....... ] 38.75% done
Progress: [....... ] 39.06% done
Progress: [....... ] 39.38% done
Progress: [....... ] 39.69% done
Progress: [........ ] 40.00% done
Progress: [........ ] 40.31% done
Progress: [........ ] 40.62% done
Progress: [........ ] 40.94% done
Progress: [........ ] 41.25% done
Progress: [........ ] 41.56% done
Progress: [........ ] 41.88% done
Progress: [........ ] 42.19% done
Progress: [........ ] 42.50% done
Progress: [........ ] 42.81% done
Progress: [........ ] 43.12% done
Progress: [........ ] 43.44% done
Cannot infer shapes or values for node "Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/range_2".
Range operation Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/range_2 with dynamic inputs is not supported
It can happen due to bug in custom shape infer function <function Range.infer at 0x7fe6ba494048>.
Or because the node inputs have incorrect values/shapes.
Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
Run Model Optimizer with --log_level=DEBUG for more information.
[ ANALYSIS INFO ] Your model looks like TensorFlow Object Detection API Model.
Check if all parameters are specified:
--tensorflow_use_custom_operations_config
--tensorflow_object_detection_api_pipeline_config
--input_shape (optional)
--reverse_input_channels (if you convert a model to use with the Inference Engine sample applications)
Detailed information about conversion of this model can be found at
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html
Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "Postprocessor/BatchMultiClassNonMaxSuppression/MultiClassNonMaxSuppression/range_2" node.
For more information please refer to Model Optimizer FAQ, question #38. (https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html?question=38#question-38)
Hide Details```

Is there a way to get around this? Is it possible to have RGBD based object detection models in OpenVINO? Thanks!

[IntelSupport]

Hi SaifVazir,

Greetings to you. Have you tried to convert your model using OpenVINO Model Optimizer directly? Is that the last message that you got from your terminal? It seems like no error arises, could you share your model so that we can test it on our side. Meanwhile, please try to follow the instruction from Converting TensorFlow* Object Detection API models documentation and see if this can solve your issue.

Regards,

Aznie

[SaifVazir]

I tried using the model optimizer directly using the command:

python mo_tf.py --input_model "C:\Users\saif\Downloads\inference_graph_mbnetv2_300_depth\mbnetv2_depth.pb" --input_shape [1,300,300,4] --transformations_config ".\extensions\front\tf\ssd_support_api_v1.15.json" --tensorflow_object_detection_api_pipeline_config "C:\Users\saif\Downloads\inference_graph_mbnetv2_300_depth\pipeline.config"

I get the same error as before. I'm attaching the model and config files for you to test it out. Thanks for helping me! 

[IntelSupport]

Hi SaifVazir,

Thanks for your model. We will investigate this and update you with further information soon.

Regards,

Aznie

[IntelSupport]

Hi SaifVazir,

We apologize for the delay in replying to you. Thank you for your patience, after investigating the trained model, it seems that you are using the TF v1.5 to train your model. TF v1.5 is quite an outdated version of TF API.

For our suggestion based on these documents, it is advised to use a validated version of TF Object Detection API which is version 1.14, 1.15, 2.x to trained the custom model. You can freeze the model using TF OD API version 1.14 or 1.15 or 2.x, and then try to convert using model optimizer gain based on the command below

1. python3 mo.py --framework=tf --data_type=FP32 --output_dir=<INSTALL_DIR>/openvino_2021.3.394/deployment_tools/open_model_zoo/tools/downloader/public/ssdlite_mobilenet_v2/FP32 --model_name=ssdlite_mobilenet_v2 --reverse_input_channels '--input_shape=[1,300,300,3]' --input=image_tensor --output=detection_classes,detection_boxes,detection_scores,detection_multiclass_scores --tensorflow_object_detection_api_pipeline_config=/home/user/Desktop/inference_graph_mbnetv2_300_depth/inference_graph_mbnetv2_300_depth/pipeline.config --input_model=/home/user/Desktop/inference_graph_mbnetv2_300_depth/inference_graph_mbnetv2_300_depth/mbnetv2_depth.pb --transformations_config=/<INSTALL_DIR>/openvino_2021.3.394/deployment_tools/model_optimizer/extensions/front/tf/ssd_support_api_v1.14.json

Hope this information help.

Regards,

Aznie

[IntelSupport]

Hi SaifVazir,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question. 

Regards,

Aznie