Model Optimizer About Faster RCNN with VGG16 as Feature Extractor

[zhang__chunyan]

Hi,

Does OPENVINO support Faster RCNN models (tensorflow) that not trained by TF Object Detection API?

For example, https://github.com/endernewton/tf-faster-rcnn

Zhang

[HemanthKum_G_Intel]

Hi Zhang,

OpenVINO supports Tensorflow, Caffe, ONNX, Mxnet, Kaldi frameworks. If you use custom API, finally it should adhere to the supported layers in the supported frameworks. For a couple of custom layers, one can write definitions in Model Optimizer. The following link provides you the layers from respective frameworks supported in OpenVINO -

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Supported_Frameworks_Layers.html

The link you shared says that tf-faster-rcnn is deprecated and pointing to https://github.com/tensorpack/tensorpack/tree/master/examples/FasterRCNN which has Tensorflow based implementation. 

OpenVINO release notes say that faster_rcnn_inception_v2_coco and faster_rcnn_resnet101_coco with Tensorflow implementations are available for download through model downloader script. 

[zhang__chunyan]

Hi， Hemanth Kumar G

Thanks for your reply.

I have trained a Faster_RCNN_Resnet50 with TF Object Detection API, when I convert the model,

python mo_tf.py --input_model ..\frozen_inference_graph.pb --tensorflow_use_custom_operations_config ..\faster_rcnn_support_api_v1.10.json --tensorflow_object_detection_api_pipeline_config ..\pipeline.config

I got this error:

Model Optimizer version:        2019.1.1-83-g28dfbfd
[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
Specify the "--input_shape" command line parameter to override the default shape which is equal to (600, 600).
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

The OpenVINO version of mine is: 2019.1.148

Then I tried to convert the faster_rcnn_resnet50_coco model download from tensorflow model zoo:

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

I got the same error as before. Could you help me about this?

Thank you.

Zhang

[Shubha_R_Intel]

Dear zhang, chunyan,

Please use the faster_rcnn_support_api_v1.13.json from the attached zip file.

Hope it helps

Thanks,

Shubha

[zhang__chunyan]

Dear Shubha,

Thanks for your reply,

I have download the faster_rcnn_support_api_v1.13.json and replace the old one, but I still get the same error, part of the DEBUG log is as follows:

[ 2019-07-12 09:17:32,403 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "1.0"
[ 2019-07-12 09:17:32,404 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "1.0"
[ 2019-07-12 09:17:32,404 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "0.0" for path ".*_nms_score_threshold"
[ 2019-07-12 09:17:32,404 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "0.7" for path ".*_nms_iou_threshold"
[ 2019-07-12 09:17:32,404 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "300" for path ".*_max_proposals"
[ 2019-07-12 09:17:32,405 ] [ DEBUG ] [ pipeline_config:115 ]  Found value "14" for path "initial_crop_size"
[ 2019-07-12 09:17:32,405 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "SOFTMAX" for path ".*/score_converter"
[ 2019-07-12 09:17:32,406 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "0.0" for path ".*/batch_non_max_suppression/score_threshold"
[ 2019-07-12 09:17:32,406 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "0.6" for path ".*/batch_non_max_suppression/iou_threshold"
[ 2019-07-12 09:17:32,407 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "100" for path ".*/batch_non_max_suppression/max_detections_per_class"
[ 2019-07-12 09:17:32,407 ] [ DEBUG ] [ pipeline_config:122 ]  Found value "300" for path ".*/batch_non_max_suppression/max_total_detections"
[ 2019-07-12 09:17:32,407 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "10.0"
[ 2019-07-12 09:17:32,407 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "10.0"
[ 2019-07-12 09:17:32,408 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "5.0"
[ 2019-07-12 09:17:32,408 ] [ DEBUG ] [ pipeline_config:126 ]  There is no value path "None". Set default value "5.0"
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.
[ 2019-07-12 09:17:32,425 ] [ DEBUG ] [ main:318 ]  Traceback (most recent call last):
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 167, in apply_replacements
    replacer.find_and_replace_pattern(graph)
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\front\tf\replacement.py", line 89, in find_and_replace_pattern
    self.replace_sub_graph(graph, match)
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\front\common\replacement.py", line 131, in replace_sub_graph
    new_sub_graph = self.generate_sub_graph(graph, match)  # pylint: disable=assignment-from-no-return
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\extensions\front\tf\ObjectDetectionAPI.py", line 581, in generate_sub_graph
    format(proposal_nodes_ids))
mo.utils.error.Error: Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\main.py", line 312, in main
    return driver(argv)
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\main.py", line 263, in driver
    is_binary=not argv.input_model_is_text)
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\pipeline\tf.py", line 127, in tf2nx
    class_registration.apply_replacements(graph, class_registration.ClassType.FRONT_REPLACER)
  File "C:\Intel\openvino_2019.1.148\deployment_tools\model_optimizer\mo\utils\class_registration.py", line 184, in apply_replacements
    )) from err
mo.utils.error.Error: Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

Could you please help me about this error?

Thank you,

Zhang