object_detection_demo_yolov3_async.py : xmin = int((x - w / 2) * w_scale) ValueError: cannot convert

[CBell1]

Hello,

 when I run object_detection_demo_yolov3_async.py with custom trained yolov3 tiny model, I'm experiencing the following error:

-----------------------------------

Error:

/opt/intel/openvino/deployment_tools/open_model_zoo/demos/python_demos/object_detection_demo_yolov3_async$ python3 object_detection_demo_yolov3_async.py -i 'cam' -m /home/user/Documents/Progetti/AI/ML_DL/YOLOv3_Keras_Cust_Model/YOLOv4/worker_safety_yolov3_tiny/IR_FP16/frozen_yolov3-tiny_workersafety_final_416.xml -d MYRIAD
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.7 of module 'openvino.inference_engine.ie_api' does not match runtime version 3.6
return f(*args, **kwds)
[ INFO ] Creating Inference Engine...
[ INFO ] Loading network
[ INFO ] Preparing inputs
[ INFO ] Loading model to the plugin
[ INFO ] Starting inference...
To close the application, press 'CTRL+C' here or switch to the output window and press ESC key
To switch between sync/async modes, press TAB key in the output window
object_detection_demo_yolov3_async.py:274: DeprecationWarning: shape property of IENetLayer is deprecated. Please use shape property of DataPtr instead objects returned by in_data or out_data property to access shape of input or output data on corresponding ports
out_blob = out_blob.reshape(net.layers[net.layers[layer_name].parents[0]].shape)
[ INFO ] Layer detector/yolo-v3-tiny/Conv_12/BiasAdd/YoloRegion parameters:
[ INFO ] classes : 2
[ INFO ] num : 3
[ INFO ] coords : 4
[ INFO ] anchors : [10.0, 14.0, 23.0, 27.0, 37.0, 58.0]
/usr/lib/python3/dist-packages/apport/report.py:13: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp, stat
Traceback (most recent call last):
File "object_detection_demo_yolov3_async.py", line 359, in <module>
sys.exit(main() or 0)
File "object_detection_demo_yolov3_async.py", line 280, in main
args.prob_threshold)
File "object_detection_demo_yolov3_async.py", line 151, in parse_yolo_region
h_scale=orig_im_h, w_scale=orig_im_w))
File "object_detection_demo_yolov3_async.py", line 100, in scale_bbox
xmin = int((x - w / 2) * w_scale)
ValueError: cannot convert float NaN to integer

--------------------------------------

Environment:

Ubuntu 18.04 LTS

OpenVino 2020 R3 LTS

Python3.6

TF 1.11.0

I trained the Darknet Model using Intel CPU and I converted the wheights and PB to IR following the following instructions:

https://docs.openvinotoolkit.org/2020.3/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html

I suppose the problem is about .json file used with mo_tf.py to convert .pb to IR, anyway, eventually I don't see the issue.

Attached you can find the .cfg used to train the Model and the .json file used with mo_tf.py

Important Note: using the same data, procedure to train yolov3 model and to convert wheights and pb with NOT tiny, everything is working properly.

Please, do you have suggestions

Thank you

[Iffa_Intel]

Greetings,

We had previously trained a yolov3-tiny on a custom dataset and the .json config has one small change when compared to yours.We suggest that you modify your json to match the following. The only difference is the "mask" values. Basically you would need to remove [3,4,5] and see if it makes a difference.

[

 {

  "id": "TFYOLOV3",

  "match_kind": "general",

  "custom_attributes": {

   "mask": [0, 1, 2],

   "anchors": [10, 14, 23, 27, 37, 58, 81, 82, 135, 169, 344, 319],

   "classes": 2,

   "num": 6,

   "jitter": 0.3,

   "ignore_thresh": 0.7,

   "truth_thresh": 1,

   "random": 1,

   "coords": 4,

   "entry_points": ["detector/yolo-v3-tiny/Reshape", "detector/yolo-v3-tiny/Reshape_4"]

  }

 }

]

Sincerely,

Iffa

[CBell1]

Hi Iffa,

I modified the .json following the tips, but running the model optimizer mo_tf.py the output is the following error:

[ WARNING ] Use of deprecated cli option --tensorflow_use_custom_operations_config detected. Option use in the following releases will be fatal. Please use --transformations_config cli option instead
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: /home/usr/Documents/Progetti/AI/ML_DL/YOLOv3_Keras_Cust_Model/YOLOv4/worker_safety_yolov3_tiny/frozen_yolov3-tiny_workersafety_Intel_416_final.pb
- Path for generated IR: /opt/intel/openvino_2020.3.341/deployment_tools/model_optimizer/.
- IR output name: frozen_yolov3-tiny_workersafety_Intel_416_final
- Log level: ERROR
- Batch: 1
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: Not specified, inherited from the model
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP16
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: False
- Reverse input channels: False
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: /opt/intel/openvino/deployment_tools/model_optimizer/extensions/front/tf/workersafety_yolo_v3_tiny_Intel.json
Model Optimizer version:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
[ ERROR ] Cannot infer shapes or values for node "detector/yolo-v3-tiny/Conv_12/BiasAdd/YoloRegion".
[ ERROR ] object of type 'int' has no len()
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function RegionYoloOp.regionyolo_infer at 0x7fae23e43d90>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ANALYSIS INFO ] Your model looks like YOLOv3 Model.
To generate the IR, provide TensorFlow YOLOv3 Model to the Model Optimizer with the following parameters:
--input_model <path_to_model>/yolo_v3.pb
--batch 1
--tensorflow_use_custom_operations_config <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json
Detailed information about conversion of this model can be fount at
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "detector/yolo-v3-tiny/Conv_12/BiasAdd/YoloRegion" node.
For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

Please, let me know your opinion

Thank you for your support

[Iffa_Intel]

We would really appreciate it if you could provide us the complete command that you had used with the model optimizer.

The key to finding the solution might be there.

Sincerely,

Iffa

[CBell1]

Sure, please, see the command below and .json used attached:

sudo python3 mo_tf.py --input_model frozen_yolov3-tiny_workersafety_Intel_416_final.pb --tensorflow_use_custom_operations_config /opt/intel/openvino/deployment_tools/model_optimizer/extensions/front/tf/workersafety_yolo_v3_tiny_Intel.json --batch 1 --data_type FP16

To convert tf .weights to .pb, I followed the following great document you shared in the past that works fine with also other projects:

https://docs.openvinotoolkit.org/latest/openvino_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html

Please, let me know you opinion / suggestions

Thank you

[Iffa_Intel]

You need to specify "--input shape" parameter, e.g. >python3 mo_tf.py --input_model model.pb ^

--tensorflow_use_custom_operations_config yolo_v3_tiny.json --input_shape "[1, 608, 608, 3]"

Plus, it's better to replace --tensorflow_use_custom_operations_config with --transformations_config

Sincerely,

Iffa

[CBell1]

Hi Iffa

unfortunately, I still have errors using --input_shape [-1,416,416,3]  (as identify by summarize_graph.py tool)  or [1,416,416,3] (all positive):

/opt/intel/openvino/deployment_tools/model_optimizer$ sudo python3 mo_tf.py --input_model /home/user/Documents/Progetti/AI/ML_DL/YOLOv3_Keras_Cust_Model/YOLOv4/worker_safety_yolov3_tiny/frozen_yolov3-tiny_workersafety_Intel_416_final.pb --transformations_config /opt/intel/openvino/deployment_tools/model_optimizer/extensions/front/tf/workersafety_yolo_v3_tiny_Intel.json --input_shape [-1,416,416,3] --data_type FP16
Model Optimizer arguments:
Common parameters:
- Path to the Input Model: /home/user/Documents/Progetti/AI/ML_DL/YOLOv3_Keras_Cust_Model/YOLOv4/worker_safety_yolov3_tiny/frozen_yolov3-tiny_workersafety_Intel_416_final.pb
- Path for generated IR: /opt/intel/openvino_2020.3.341/deployment_tools/model_optimizer/.
- IR output name: frozen_yolov3-tiny_workersafety_Intel_416_final
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: Not specified, inherited from the model
- Output layers: Not specified, inherited from the model
- Input shapes: [-1,416,416,3]
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP16
- Enable fusing: True
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: False
- Reverse input channels: False
TensorFlow specific parameters:
- Input model in text protobuf format: False
- Path to model dump for TensorBoard: None
- List of shared libraries with TensorFlow custom layers implementation: None
- Update the configuration file with input/output node names: None
- Use configuration file used to generate the model with Object Detection API: None
- Use the config file: None
Model Optimizer version:
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:523: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:524: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/dtypes.py:532: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
[ ERROR ] Shape [ -1 416 416 3] is not fully defined for output 0 of "inputs". Use --input_shape with positive integers to override model input shapes.
[ ERROR ] Cannot infer shapes or values for node "inputs".
[ ERROR ] Not all output shapes were inferred or fully defined for node "inputs".
For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #40.
[ ERROR ]
[ ERROR ] It can happen due to bug in custom shape infer function <function Parameter.infer at 0x7f12d60da598>.
[ ERROR ] Or because the node inputs have incorrect values/shapes.
[ ERROR ] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
[ ERROR ] Run Model Optimizer with --log_level=DEBUG for more information.
[ ANALYSIS INFO ] Your model looks like YOLOv3 Model.
To generate the IR, provide TensorFlow YOLOv3 Model to the Model Optimizer with the following parameters:
--input_model <path_to_model>/yolo_v3.pb
--batch 1
--tensorflow_use_custom_operations_config <OPENVINO_INSTALL_DIR>/deployment_tools/model_optimizer/extensions/front/tf/yolo_v3.json
Detailed information about conversion of this model can be fount at
https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_convert_model_tf_specific_Convert_YOLO_From_Tensorflow.html
[ ERROR ] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "inputs" node.
For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

Please, let me know your opinion

Thank you

[Iffa_Intel]

In that case, can you share your model and files (.weights and .label) for us to try on our end.

You can share it privately if you don't want to attach it here.

Sincerely,

Iffa

[CBell1]

Hi Iffa,

as requested, please, see all attached files.

Thanks

[Iffa_Intel]

Noted, we will further investigate & update asap.

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

We are able to reproduce your issue with the CPU plugin but the model runs on MYRIAD/GPU. However, We don't see any detections. Could you give us a sample image/video for us to test? This may be a bug on the CPU plugin.

Sincerely,

Iffa

[CBell1]

Hi Iffa,

thank you for your email.

Did you proporly convert the .pb to IR ?

You can use the images from Kaggle, links below:

https://www.kaggle.com/andrewmvd/face-mask-detection

https://www.kaggle.com/andrewmvd/hard-hat-detection

Please, let me know

Thank you

[JesusE_Intel]

Hi CBell1,

I was able to convert your model from weight -> .pb -> .xml/.bin. However, there is a RuntimeWarning: invalid value encountered in sqrt scale = 1. / np.sqrt(variance.data.get_value() + eps) from the model optimizer and ValueError when inferencing using the demo. Could you please verify your weight model is valid and can detect objects directly in DarkNet (assuming the model was trained on DarkNet)?

To convert from weight to .pb, I followed the instructions here.

To convert .pb to IR (.xml/.bin):

python3 /opt/intel/openvino_2021/deployment_tools/model_optimizer/mo.py --framework tf --reverse_input_channels --input_model frozen_darknet_yolov3_model.pb --batch 1 --transformations_config yolo_v3_tiny.json --data_type FP32

I tested the above steps with your model and the yolo_v3_tiny model available in pjreddie repository. These steps worked for model from pjreddie repo but not with your model. What base model did you use?

Regards,

Jesus

[CBell1]

Hi Jesus,

you recreated the similar error I have.

I followed the same steps you sent me (only difference is FP16).

Exactly, I used darknet53.conv.74 downloaded from pjreddie darknet

I cannot attach this file due to is ~150MB compressed.

Can the darknet53.conv.74 be corrupted?

Or with Yolo v3 Tiny should I use another model?

Thank you

[CBell1]

Hi Jesus, Hi Iffa,

I found a possible solution.

You are right there is something doesn't work with the pre-trained Weights model.

These are the official Yolo v3 darknet models:

• darknet53.conv.74
• yolov3-tiny.conv.11

With tiny I should use the second model instead the first that it's built for yolov3 (no tiny).

I test the yolov3-tiny.conv.11 with my custom solution and I'll keep you informed

Best Regards

[CBell1 · Accepted solution]

Hi Jesus, Hi Iffa,

problem solved using yolov3-tiny.conv.11

The object detection yolo v3 with my customization run properly recognizing masks

Thank you for your support

[JesusE_Intel]

Hi CBell1,

Glad you were able to solve your issue! Intel will no longer monitor this thread since this issue has been resolved. If you need any additional information from Intel, please submit a new question. 

Regards,

Jesus

[haviernick]

You can avoid this with a mask method. Note first that in python NaN is defined as the number which is not equal to itself:

>float('nan') == float('nan')
False

It might be worth avoiding use of np.NaN altogether. NaN literally means "not a number", and it cannot be converted to an integer. In general, Python prefers raising an exception to returning NaN, so things like sqrt(-1) and log(0.0) will generally raise instead of returning NaN. However, you may get this value back from some other library. From v0.24, you actually can. Pandas introduces Nullable Integer Data Types which allows integers to coexist with NaNs. Also, even at the lastest versions of pandas if the column is object type you would have to convert into float first, something like:

df['column_name'].astype(np.float).astype("Int32")

NB: You have to go through numpy float first and then to nullable Int32, for some reason.