Custom tensorflow Faster RCNN inference issues

[Talbi__Ahmed]

Hello, 
I am trying to run a custom trained Faster RCNN (trained on my own data) But I am getting the following error:

 

(tensorflow_last) root@DEV11:~/inference_engine_samples_build/intel64/Release# ./object_detection_sample_ssd -i /dev/video0 -m /home/dev11/Desktop/Ahmed/train_fasterRCNN_ResNet_osr_2/frozen_inference_graph.xml -d MYRIAD
[ INFO ] InferenceEngine: 
    API version ............ 1.4
    Build .................. 19154
Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     /dev/video0
[ INFO ] Loading plugin

    API version ............ 1.5
    Build .................. 19154
    Description ....... myriadPlugin
[ INFO ] Loading network files:
    /home/ciates-dev11/Desktop/Ahmed/train_fasterRCNN_ResNet_osr_2/frozen_inference_graph.xml
    /home/ciates-dev11/Desktop/Ahmed/train_fasterRCNN_ResNet_osr_2/frozen_inference_graph.bin
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Preparing output blobs
[ INFO ] Loading model to the plugin
[ ERROR ] AssertionFailed: newDims[newPerm] == 1
 

The same pretrained models from the tensorflow Modelzoo are working, but after training I am always getting this error msg. 
I already tried the Inception_v2 and the resnet50. I also tried to keep the same number of classes in the pipeline config file but nothing changed. 

does anyone already tried to train it own (tensorflow FasterRCNN)model?  
P.S: Already tried the Caffe VGG FasterRcnn (both pretrained and custom models) and was able to load both of them.

[Keyes__Mike]

I'm also getting this exact error with a retrained faster_rcnn_inception_v2_coco Tensorflow model.The pretrained model works fine. If anyone has a clue how to get past this error I'd appreciate hearing it!  Thanks

[nikos1]

Hi Ahmed, 

> /train_fasterRCNN_ResNet_osr_2/frozen_inference_graph.xml -d MYRIAD

Are you running on NCS (MYRIAD plug-in) ? Is -d CPU okay?

Just wondering if TF Faster RCNN is now supported in MYRIAD?

According to documentation in  https://software.intel.com/en-us/articles/OpenVINO-RelNotes

TensorFlow Faster R-CNNs and Mask R-CNNs are supported on CPU only and with batch size 1. 

nikos

[Talbi__Ahmed]

HI Nikos, 
I am trying to make it run with on the NCS.
I know about the documentation but I dont know if they didnt change it for the latest release. 
Plus the pretrained models run, so I dont see any reason why the trained ones dont. In fact, I think this due to version of tensorflow used. See the pretrained models are generated with tf 1.8 but the actual obj det api only supports tf 1.9 or above, I got my hand on an old version of the api retrained the model with tf 1,8 and tried to convert again but now i am having another type of issues, relative to the json conversion file. 

 

Traceback (most recent call last):
  File "/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/mo/main.py", line 325, in main
    return driver(argv)
  File "/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/mo/main.py", line 267, in driver
    mean_scale_values=mean_scale)
  File "/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/mo/pipeline/tf.py", line 251, in tf2nx
    class_registration.apply_replacements(graph, class_registration.ClassType.FRONT_REPLACER)
  File "/opt/intel/computer_vision_sdk_2018.5.445/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 121, in apply_replacements
    )) from err
mo.utils.error.Error: Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

Ahmed

[nikos1]

Thanks Ahmed! Good to know.

[Khandelwal__Nikhil]

I am also facing this issue "RuntimeError: AssertionFailed: newDims[newPerm] == 1" with MYRIAD plugin.

But i am able to inference with CPU and GPU plugin with the same IR

[Sangam__AI]

I have created custom object detection tensorflow model using FasterRCNN and trying to create .bin and .xml file using open vino. While I was running the below step

sudo python3 mo_tf.py --input_model /home/ai/workspaceOV/cat_dog_model/dog_cat_inference_graph/frozen_inference_graph.pb --tensorflow_use_custom_operations_config /opt/intel/computer_vision_sdk_2018.5.455/deployment_tools/model_optimizer/extensions/front/tf/faster_rcnn_support.json --tensorflow_object_detection_api_pipeline_config /home/ai/workspaceOV/cat_dog_model/dog_cat_inference_graph/pipeline.config

I am getting the following error

odel Optimizer version:     1.5.12.49d067a0
[ WARNING ] Model Optimizer removes pre-processing block of the model which resizes image keeping aspect ratio. The Inference Engine does not support dynamic image size so the Intermediate Representation file is generated with the input image size of a fixed size.
Specify the "--input_shape" command line parameter to override the default shape which is equal to (600, 600).
The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
[ ERROR ]  Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work.

I searched a lot about it on google and find one solution. I have implemented it. Solution which i found is to find CropAndResize Operation from graph file in tensorflow and place that in faster_rcnn_support.json. I did it. Please visit the below link  for CropAndResize Operation

https://drive.google.com/file/d/1DnUF54c1JJ8yjV_xD5E4xFIyJxSJoUGF/view?usp=sharing

I hope anyone can help so that i can work on it.

[Stankov__Stanko]

Hello Sangam, AI

Can you provide an example of the placed " CropAndResize Operation " in  faster_rcnn_support.json?

I am receiving the exact same error - 

Exception occurred during running replacer "ObjectDetectionAPIDetectionOutputReplacement" (<class 'extensions.front.tf.ObjectDetectionAPI.ObjectDetectionAPIDetectionOutputReplacement'>): Found the following nodes '[]' with name 'proposals' but there should be exactly 1. Looks like ObjectDetectionAPIProposalReplacement replacement didn't work."

"

Thank you for your time!