Model Optimizer Convert Faster RCN NAS COCO Model Error

[Sack__Benjamin]

Model: http://download.tensorflow.org/models/object_detection/faster_rcnn_nas_coco_2018_01_28.tar.gz

Command: 

python3 mo_tf.py 
--input_model /home/benjamins/Downloads/faster_rcnn_nas_coco_2018_01_28/frozen_inference_graph.pb 
--model_name Person_Detection 
--output_dir /home/benjamins/Desktop/Models/Out/
 --reverse_input_channels 
--tensorflow_object_detection_api_pipeline_config /home/benjamins/Downloads/faster_rcnn_nas_coco_2018_01_28/pipeline.config 
--tensorflow_use_custom_operations_config/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support.json

Error:

[ ERROR ]  Cannot infer shapes or values for node "SecondStageFeatureExtractor/cell_12/strided_slice".
[ ERROR ]  
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_strided_slice_infer at 0x7f43633ab048>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Stopped shape/value propagation at "SecondStageFeatureExtractor/cell_12/strided_slice" node. 

Converting the Tensorflow Model with the Model Optimizer is not working

[Shubha_R_Intel]

Dear Benjamin:

Did you get your Faster RCNN from this list here ? 

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

Please choose one of the tar.gz from the list above.

Thanks for using OpenVino !

Shubha

[Sack__Benjamin]

Dear Shubha:

With the Faster RCNN Model from the link above I'm getting the same error .

python3 mo_tf.py --input_model /home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/frozen_inference_graph.pb --model_name Person_Detection --output_dir /home/benjamins/Desktop/Models/Out/ --reverse_input_channels --tensorflow_object_detection_api_pipeline_config /home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/pipeline.config --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support.json

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/frozen_inference_graph.pb
	- Path for generated IR: 	/home/benjamins/Desktop/Models/Out/
	- IR output name: 	Person_Detection
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	True
TensorFlow specific parameters:
	- Input model in text protobuf format: 	False
	- Offload unsupported operations: 	False
	- Path to model dump for TensorBoard: 	None
	- List of shared libraries with TensorFlow custom layers implementation: 	None
	- Update the configuration file with input/output node names: 	None
	- Use configuration file used to generate the model with Object Detection API: 	/home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/pipeline.config
	- Operations to offload: 	None
	- Patterns to offload: 	None
	- Use the config file: 	/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support.json
Model Optimizer version: 	1.5.12.49d067a0
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
The graph output nodes "num_detections", "detection_boxes", "detection_classes", "detection_scores" have been replaced with a single layer of type "Detection Output". Refer to IR catalogue in the documentation for information about this layer.
[ ERROR ]  Cannot infer shapes or values for node "SecondStageFeatureExtractor/cell_12/strided_slice".
[ ERROR ]  
[ ERROR ]  
[ ERROR ]  It can happen due to bug in custom shape infer function <function tf_strided_slice_infer at 0x7f213b2ac1e0>.
[ ERROR ]  Or because the node inputs have incorrect values/shapes.
[ ERROR ]  Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ]  Run Model Optimizer with --log_level=DEBUG for more information.
[ ERROR ]  Stopped shape/value propagation at "SecondStageFeatureExtractor/cell_12/strided_slice" node. 
 For more information please refer to Model Optimizer FAQ (<INSTALL_DIR>/deployment_tools/documentation/docs/MO_FAQ.html), question #38.

Thanks and Regards,

Benjamin

[Shubha_R_Intel]

Dear Benjamin,

there is a faster_rcnn_support.json, a faster_rcnn_support_api_v1.7.json, and a faster_rcnn_support_api_v1.10.json. Are you sure you are using the correct version ? It's based on the Tensorflow API version.

Thanks,

Shubha

[Sack__Benjamin]

Dear Shubha,

python3 -c 'import tensorflow as tf; print(tf.__version__)'
1.12.0

I'm using faster_rcnn_support.json, because faster_rcnn_support_api_v1.10.json is not available in the folder.

Thanks and Regards,

Benjamin

[Shubha_R_Intel]

Dearest Benjamin,

we are about to release a new version of OpenVino any day now and guess what ? In this release you will definitely find faster_rcnn_support_api_v1.10.json under C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\extensions\front\tf (for Windows) or a similar path for Ubuntu.

Shubha

[Sack__Benjamin]

Dear Shubha,

I'm getting still a error on converting the model

python3 mo_tf.py --input_model /home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/frozen_inference_graph.pb --model_name Person_Detection --output_dir /home/benjamins/Desktop/Models/Out/ --reverse_input_channels --tensorflow_object_detection_api_pipeline_config /home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/pipeline.config --tensorflow_use_custom_operations_config /opt/intel/openvino_2019.1.094/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support_api_v1.10.json 
Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	/home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/frozen_inference_graph.pb
	- Path for generated IR: 	/home/benjamins/Desktop/Models/Out/
	- IR output name: 	Person_Detection
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	False
	- Reverse input channels: 	True
TensorFlow specific parameters:
	- Input model in text protobuf format: 	False
	- Path to model dump for TensorBoard: 	None
	- List of shared libraries with TensorFlow custom layers implementation: 	None
	- Update the configuration file with input/output node names: 	None
	- Use configuration file used to generate the model with Object Detection API: 	/home/benjamins/Downloads/new/faster_rcnn_nas_coco_2018_01_28/pipeline.config
	- Operations to offload: 	None
	- Patterns to offload: 	None
	- Use the config file: 	/opt/intel/openvino_2019.1.094/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support_api_v1.10.json
Model Optimizer version: 	2019.1.0-341-gc9b66a2
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

Thanks and Regards,

Benjamin

[Shubha_R_Intel]

Dearest Benjamin, the document below gives detailed rules about which *.json you can use with which Tensorflow API version. Are you sure you're following these rules correctly ? And most importantly are you feeding in a frozen tensorflow protobuf  to Model Optimizer ? Please see the document links below.

I repeat the rules here for your convenience:

ssd_v2_support.json — for frozen SSD topologies from the models zoo

faster_rcnn_support.json — for frozen Faster R-CNN topologies from the models zoo

faster_rcnn_support_api_v1.7.json — for Faster R-CNN topologies trained manually using the TensorFlow* Object Detection API version 1.7.0 or higher

faster_rcnn_support_api_v1.10.json — for Faster R-CNN topologies trained manually using the TensorFlow* Object Detection API version 1.7.0 or higher

mask_rcnn_support.json — for frozen Mask R-CNN topologies from the models zoo

mask_rcnn_support_api_v1.7.json — for Mask R-CNN topologies trained manually using the TensorFlow* Object Detection API version 1.7.0 or higher up to 1.9.0 inclusively

mask_rcnn_support_api_v1.11.json — for Mask R-CNN topologies trained manually using the TensorFlow* Object Detection API version 1.10.0 or higher

rfcn_support.json — for the frozen RFCN topology from the models zoo frozen with TensorFlow* version 1.9.0 or lower.

rfcn_support_api_v1.10.json — for the frozen RFCN topology from the models zoo frozen with TensorFlow* version 1.10.0 or higher.

https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_Object_Detection_API_Models.html

Freezing the Tensorflow Model

[Sack__Benjamin]

Dear Shubha,

I'm using the correct *.json for the Model.

This (faster_rcnn_support.json — for frozen Faster R-CNN topologies from the models zoo ) is the correct Version, because the Model is form the Model Zoo and a Faster R-CNN Model.

[Shubha_R_Intel]

Dear Benjamin, fair enough. Let me try it myself and i will report my findings here.

Thanks for your patience,

Shubha

[Sack__Benjamin]

Here is what you need to do and it could explain some of the problems you've been having, including your model optimizer conversion issues. Please go to your Windows Add or Remove Programs, look up OpenVino and uninstall all old versions. There's no reason to have those old versions hanging around. Then re-install 2019 R1. It seems like something is horribly messed up in your installation environment.

I run the model conversion on a Ubuntu 16.04 System with only one installed software pack 2019 R1 

[Shubha_R_Intel]

Dear Benjamin,

thanks for your patience. As promised ,I will try it myself this week.

Shubha

[Jonathan_B_Intel1]

Hi, I recreated the exact same error with openvino_2019.1.094 and with faster_rcnn_resnet101_coco_2018_01_28 and faster_rcnn_resnet50_coco_2018_01_28 downloaded from the openvino links. I also tried faster_rcnn_support faster_rcnn_support_api_v1.7 and faster_rcnn_support_api_v1.10 json files.

crop_proposals is only mentioned in:

./openvino_2019.1.094/deployment_tools/model_optimizer/extensions/front/tf/ObjectDetectionAPI.py:        crop_node = crop_op.create_node([tf_proposal_reshape_4d_node], dict(name='crop_proposals'))

maybe the name changed?

[Shubha_R_Intel]

Dear Benjamin and Jonathan,

So I got 2019 R1 Model Optimizer to generate IR using the following command:

Program Files (x86)\IntelSWTools\openvino_2019.1.087\deployment_tools\model_optimizer>python mo_tf.py --input_model c:\users\sdramani\Downloads\faster_rcnn_nas_coco_2018_01_28\frozen_inference_graph.pb  --tensorflow_use_custom_operations_config "c:\Program Files (x86)\IntelSWTools\openvino_2019.1.087\deployment_tools\model_optimizer\extensions\front\tf\faster_rcnn_support.json" --tensorflow_object_detection_api_pipeline_config "c:\users\sdramani\Downloads\faster_rcnn_nas_coco_2018_01_28\pipeline.config"

I tried the others too faster_rcnn_support_api_v1.7.json and faster_rcnn_support_api_v1.10.json but they failed, which leads me to believe that the TensorFlow* Object Detection API  used is older than v.1.7 . If you look at the date of the model, it's pretty old (about 1.5 years) so the fact that an older API was used is quite believable. 

Someone on github dldt however is having issues running inference on the generated IR. I am still investigating this.

https://github.com/opencv/dldt/issues/128

Thanks for your patience,

Shubha

[Jonathan_B_Intel]

Hi Subha,

The problem does not occur when it runs on the pretrained network before training so it looks like it is a change when running the object training pipeline that alters the crop proposals. Have you had any luck fixing the bug?

regards

Jonathan

[Jonathan_B_Intel1]

Hi Subha,

still having problems with R9.1.114 did you get a chance to see what is happening to the crop proposal layer?

regards

Jonathan

[Shubha_R_Intel]

Dear Johnathan, 

We have a fix for the crop proposal error issue. Please see attached. the crop proposal error happens when you attempt to convert a custom-trained Tensorflow Object Detection API model via Model Optimizer. 

Please download the latest 2019R1.1. Many improvements have been made.

Thanks,

Shubha

[RDeBo]

Trying it with the .zip above results in : 
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

on a custom object detector for me still.

Has it fixed the issue for anyone else? 

[RDeBo]

Still suffering the same issue with v1.13 here.

[RDeBo]

Hi,

I've tried this now with pets_v2 and faster_rcnn_coco_v2 (downloaded from list of approved networks in documentation).

It still results in : 

  - Use the config file:  C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\extensions\front\tf\faster_rcnn_support_api_v1.13.json
Model Optimizer version:        2019.1.1-83-g28dfbfd
[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
Specify the "--input_shape" command line parameter to override the default shape which is equal to (600, 600).
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'crop_proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

when running : 

python mo_tf.py --data_type=FP32 --tensorflow_object_detection_api_pipeline_config "pipeline.config" --tensorflow_use_custom_operations_config "C:\Program Files (x86)\IntelSWTools\openvino\deployment_tools\model_optimizer\extensions\front\tf\faster_rcnn_support_api_v1.13.json" --input_model "frozen_inference_graph.pb"

Any ideas? Thanks a lot.

[RDeBo]

Update : with Tensorflow v1.13.1, use the faster_rcnn_support.json, not version 1.13. This fixes the issue.