Trouble exporting faster rcnn model to openvino IR format

[Bharat_Sharma]

Hi,

I am facing an issue similar to mentioned in this post - 
https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/Problems-using-Faster-RCNN-TFODAPI-converted/m-p/1365158#M26709

I am trying to convert a custom trained faster rcnn model to openvino IR format and it keeps failing. I downloaded the pretrained model from here - 
https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf2_detection_zoo.md

I am using openvino version -

openvino 2021.4.2
openvino-dev 2021.4.2

The python script that is run is this - 

python env/lib/python3.8/site-packages/mo_tf.py --saved_model_dir=exported_models/faster_rcnn/saved_model --output_dir=openvino/faster_rcnn --transformations_config=env/lib/python3.8/site-packages/mo/extensions/front/tf/faster_rcnn_support_api_v2.0.json --tensorflow_object_detection_api_pipeline_config=exported_models/faster_rcnn/pipeline.config --input_shape="[1, 800, 1333, 3]"

When i use faster_rcnn_support_api_v2.0.json as the transformation config and it gives me this error - 

[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
[ WARNING ]  The model resizes the input image keeping aspect ratio with min dimension 800, max dimension 1333. The provided input height 800, width 1333 is transformed to height 1333, width 1333.
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIPreprocessor2Replacement (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIPreprocessor2Replacement'>)":
[ ERROR ]  Traceback (most recent call last):
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 276, in apply_transform
    replacer.find_and_replace_pattern(graph)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/front/tf/replacement.py", line 36, in find_and_replace_pattern
    self.transform_graph(graph, desc._replacement_desc['custom_attributes'])
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/extensions/front/tf/ObjectDetectionAPI.py", line 710, in transform_graph
    assert len(start_nodes) >= 1
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 394, in main
    ret_code = driver(argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 356, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 252, in prepare_ir
    graph = unified_pipeline(argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/pipeline/unified.py", line 13, in unified_pipeline
    class_registration.apply_replacements(graph, [
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 328, in apply_replacements
    apply_replacements_list(graph, replacers_order)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 314, in apply_replacements_list
    apply_transform(
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/logger.py", line 111, in wrapper
    function(*args, **kwargs)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 302, in apply_transform
    raise Exception('Exception occurred during running replacer "{} ({})": {}'.format(
Exception: Exception occurred during running replacer "ObjectDetectionAPIPreprocessor2Replacement (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIPreprocessor2Replacement'>)":

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

Also, when i tried using the other version of transformations config faster_rcnn_support_api_v2.4.json, i get a different error - 

[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
[ WARNING ]  The model resizes the input image keeping aspect ratio with min dimension 800, max dimension 1333. The provided input height 800, width 1333 is transformed to height 1333, width 1333.
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIProposalReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIProposalReplacement'>): The matched sub-graph contains network input node "input_tensor".

Any help to debug this would be appreciated.

Thanks,
Bharat

[Peh_Intel]

Hi Bharat,

I’ve validated that those pre-trained models, Faster R-CNN ResNet50 V1 800x1333, Faster R-CNN ResNet101 V1 800x1333 and Faster R-CNN ResNet152 V1 800x1333 from TensorFlow 2 Detection Model Zoo can be successfully converted into Intermediate Representation (IR) by using faster_rcnn_support_api_v2.0.json.

I am getting the same AssertionError if I used faster_rcnn_support_api_v2.4.json.

Please try converting model with faster_rcnn_support_api_v2.0.json again. I share faster_rcnn_support_api_v2.0.json as well.

https://drive.google.com/file/d/1SWZnhoOMLKB_eXZ9Ha3ywC79fhYLNEID/view?usp=sharing

My environment:

openvino-dev 2021.4.2

Numpy version 1.19.2

TensorFlow version 2.4.4

Regards,

Peh

[Bharat_Sharma]

Hi @Peh_Intel ,

Thank you for the response.

I still get this error message when i use the faster_rcnn_support_api_v2.0.json (tried the one provided by you as well).

[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
[ WARNING ]  The model resizes the input image keeping aspect ratio with min dimension 800, max dimension 1333. The provided input height 800, width 1333 is transformed to height 1333, width 1333.
[ ERROR ]  -------------------------------------------------
[ ERROR ]  ----------------- INTERNAL ERROR ----------------
[ ERROR ]  Unexpected exception happened.
[ ERROR ]  Please contact Model Optimizer developers and forward the following information:
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIPreprocessor2Replacement (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIPreprocessor2Replacement'>)":
[ ERROR ]  Traceback (most recent call last):
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 276, in apply_transform
    replacer.find_and_replace_pattern(graph)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/front/tf/replacement.py", line 36, in find_and_replace_pattern
    self.transform_graph(graph, desc._replacement_desc['custom_attributes'])
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/extensions/front/tf/ObjectDetectionAPI.py", line 710, in transform_graph
    assert len(start_nodes) >= 1
AssertionError

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 394, in main
    ret_code = driver(argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 356, in driver
    ret_res = emit_ir(prepare_ir(argv), argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/main.py", line 252, in prepare_ir
    graph = unified_pipeline(argv)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/pipeline/unified.py", line 13, in unified_pipeline
    class_registration.apply_replacements(graph, [
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 328, in apply_replacements
    apply_replacements_list(graph, replacers_order)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 314, in apply_replacements_list
    apply_transform(
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/logger.py", line 111, in wrapper
    function(*args, **kwargs)
  File "/home/bharat/projects/fuelmatics-lid-detection/env/lib/python3.8/site-packages/mo/mo/utils/class_registration.py", line 302, in apply_transform
    raise Exception('Exception occurred during running replacer "{} ({})": {}'.format(
Exception: Exception occurred during running replacer "ObjectDetectionAPIPreprocessor2Replacement (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIPreprocessor2Replacement'>)":

[ ERROR ]  ---------------- END OF BUG REPORT --------------
[ ERROR ]  -------------------------------------------------

[Bharat_Sharma]

I am able to convert the pre-trained model out of the box as well. But when I train the model on a custom dataset and try to convert it, it fails with the error i pasted above.

[Peh_Intel]

Hi Bharat,

Could you share the model (SavedModel format) with us for further investigation?

Regards,

Peh

[Peh_Intel]

Hi Bharat,

We have not heard back from you. Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Peh