Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

Problem with openvino inference of Faster R-CNN Inception ResNet V2 640x640

AndreyShilkov
Beginner
948 Views

Hi,
I tried to inference Faster R-CNN Inception ResNet V2 640x640 from Tensorflow2 Model Zoo. I downloaded it from openvino list of supported topologies  and used this command to generate IR:


python3 mo_tf.py --saved_model_dir /home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/saved_model --transformations_config ./extensions/front/tf/faster_rcnn_support_api_v2.0.json --tensorflow_object_detection_api_pipeline_config /home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/pipeline.config --output_dir /home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION

After receiving following output

Model Optimizer arguments:
Common parameters:
	- Path to the Input Model: 	None
	- Path for generated IR: 	/home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION
	- IR output name: 	saved_model
	- Log level: 	ERROR
	- Batch: 	Not specified, inherited from the model
	- Input layers: 	Not specified, inherited from the model
	- Output layers: 	Not specified, inherited from the model
	- Input shapes: 	Not specified, inherited from the model
	- Mean values: 	Not specified
	- Scale values: 	Not specified
	- Scale factor: 	Not specified
	- Precision of IR: 	FP32
	- Enable fusing: 	True
	- Enable grouped convolutions fusing: 	True
	- Move mean values to preprocess section: 	None
	- Reverse input channels: 	False
TensorFlow specific parameters:
	- Input model in text protobuf format: 	False
	- Path to model dump for TensorBoard: 	None
	- List of shared libraries with TensorFlow custom layers implementation: 	None
	- Update the configuration file with input/output node names: 	None
	- Use configuration file used to generate the model with Object Detection API: 	/home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/pipeline.config
	- Use the config file: 	None
	- Inference Engine found in: 	/opt/intel/openvino_2021.4.689/python/python3.6/openvino
Inference Engine version: 	2021.4.1-3926-14e67d86634-releases/2021/4
Model Optimizer version: 	2021.4.1-3926-14e67d86634-releases/2021/4
2021-09-24 09:24:48.889055: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/lib/intel64:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/external/tbb/lib:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../ngraph/lib
2021-09-24 09:24:48.889085: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
/home/stranger/mnt/TestOpenvinoConvertion/opnvn_venv/lib/python3.6/site-packages/tensorflow/python/autograph/impl/api.py:22: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
2021-09-24 09:24:51.259626: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:24:51.259798: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/lib/intel64:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../inference_engine/external/tbb/lib:/opt/intel/openvino_2021.4.689/deployment_tools/model_optimizer/mo/utils/../../../ngraph/lib
2021-09-24 09:24:51.259811: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-09-24 09:24:51.259836: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:163] no NVIDIA GPU device is present: /dev/nvidia0 does not exist
2021-09-24 09:24:51.259967: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-09-24 09:24:51.260408: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:25:14.602098: I tensorflow/core/grappler/devices.cc:69] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 0
2021-09-24 09:25:14.602217: I tensorflow/core/grappler/clusters/single_machine.cc:356] Starting new session
2021-09-24 09:25:14.602410: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:25:14.619506: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 4200000000 Hz
2021-09-24 09:25:15.382756: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:928] Optimization results for grappler item: graph_to_optimize
  function_optimizer: Graph size after: 11840 nodes (10717), 14416 edges (13286), time = 457.918ms.
  function_optimizer: function_optimizer did nothing. time = 12.462ms.

The Preprocessor block has been removed. Only nodes performing mean value subtraction and scaling (if applicable) are kept.
The graph output nodes have been replaced with a single layer of type "DetectionOutput". Refer to the operation set specification documentation for more information about the operation.
2021-09-24 09:26:55.184330: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.186958: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2021-09-24 09:26:55.211708: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.223419: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.231665: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.241611: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.275109: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.287376: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.300002: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.310105: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-09-24 09:26:55.323911: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION/saved_model.xml
[ SUCCESS ] BIN file: /home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION/saved_model.bin
[ SUCCESS ] Total execution time: 216.71 seconds. 
[ SUCCESS ] Memory consumed: 4058 MB. 

I tried to get inference using following code

import numpy as np

BIN = "/home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION/saved_model.bin"
XML = "/home/stranger/mnt/TestOpenvinoConvertion/FASTER_RCNN_MODELS/faster_rcnn_inception_resnet_v2_640x640_coco17_tpu-8/OPENVINO_CONVERTION/saved_model.xml"

net = ie_core.read_network(model=XML, weights=BIN)
exec_net = ie_core.load_network(network=net, device_name="CPU", num_requests=0)

input_image = np.zeros((1, 3, 640, 640))
net_input = {"input_tensor": input_image}
result = exec_net.infer(net_input)

and got this error:

Traceback (most recent call last):
  File "/home/stranger/mnt/TestOpenvinoConvertion/PythonScripts/test_faster_rcnn.py", line 17, in <module>
    res = exec_net.infer(input)
  File "ie_api.pyx", line 884, in openvino.inference_engine.ie_api.ExecutableNetwork.infer
  File "ie_api.pyx", line 1204, in openvino.inference_engine.ie_api.InferRequest.infer
  File "ie_api.pyx", line 1208, in openvino.inference_engine.ie_api.InferRequest.infer
RuntimeError: Proposal:proposals: Proposal operation image info input must have positive image height and width.


What is the best way to treat this issue?

0 Kudos
3 Replies
Zulkifli_Intel
Moderator
870 Views

Andrey Shilkov,

 

Thank you for reaching out to us.

 

We get a similar error as you get. Then we test the model using Benchmark Python* Tool and get the following error:

benchmark.PNG

 

We are checking this out and get back to you soon.

 

Sincerely,

Zulkifli

 

0 Kudos
Zulkifli_Intel
Moderator
814 Views

Hello Andrey Shikov,

 

Based on our findings, the model (Faster R-CNN Inception ResNet V2 640x640) was just enabling for MO for the latest version (2021.4) and our developers are working to enable this model for inferencing. The target is to be available on 2022.1, but it is yet to be confirmed.

 

Sincerely,

Zulkifli 

 

0 Kudos
Zulkifli_Intel
Moderator
775 Views

Hello Andrey Shikov,


This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.


Sincerely,

Zulkifli


0 Kudos
Reply