Stopped shape/value propagation at Conv2d node error

Friedman__Benny · ‎07-25-2019

Hi

When trying to use the model optimizer on a frozen model I created over TF2 I got this error complaining that for indices I need to provide integers.

The thing is, this operation does not get any tensor with indices. Seems like the parameter it is failing on is the padding which should be a string "SAME" or "EXPLICIT".

Here is a chunk of the output with the error:

I0725 16:47:54.095988 4391695808 infer.py:130] Partial infer for Conv1_layer/Conv

I0725 16:47:54.096096 4391695808 infer.py:131] Op: Conv2D

E0725 16:47:54.098730 4391695808 infer.py:180] Cannot infer shapes or values for node "Conv1_layer/Conv".

E0725 16:47:54.099003 4391695808 infer.py:181] arrays used as indices must be of integer (or boolean) type

E0725 16:47:54.099144 4391695808 infer.py:182]

E0725 16:47:54.099277 4391695808 infer.py:183] It can happen due to bug in custom shape infer function <function Convolution.infer at 0x125a1a510>.

E0725 16:47:54.099431 4391695808 infer.py:184] Or because the node inputs have incorrect values/shapes.

E0725 16:47:54.099596 4391695808 infer.py:185] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).

I0725 16:47:54.101786 4391695808 infer.py:196] Node "Conv1_layer/Conv" attributes: {'pb': name: "Conv1_layer/Conv"

op: "Conv2D"

input: "input"

input: "Conv1_layer/W"

attr {

key: "T"

value {

type: DT_FLOAT

}

attr {

key: "data_format"

value {

s: "NHWC"

}

attr {

key: "dilations"

value {

list {

i: 1

}

attr {

key: "explicit_paddings"

value {

list {

}

attr {

key: "padding"

value {

s: "SAME"

}

attr {

key: "strides"

value {

list {

i: 1

}

attr {

key: "use_cudnn_on_gpu"

value {

b: true

}

, '_in_ports': {0: {'control_flow': False}, 1: {'control_flow': False}, 2: {}}, '_out_ports': {0: {'control_flow': False}}, 'kind': 'op', 'name': 'Conv1_layer/Conv', 'op': 'Conv2D', 'precision': 'FP32', 'type': 'Convolution', 'infer': <function Convolution.infer at 0x125a1a510>, 'multiplication_transparent': True, 'multiplication_transparent_ports': [(0, 0), (1, 0)], 'in_ports_count': 3, 'out_ports_count': 1, 'auto_pad': 'same_upper', 'bias_addable': True, 'bias_term': False, 'dilation': array([1, 1, 1, 1]), 'stride': array([1, 1, 1, 1]), 'channel_dims': [3], 'batch_dims': [0], 'input_feature_channel': 2, 'output_feature_channel': 3, 'layout': 'NHWC', 'get_group': <function Conv2DFrontExtractor.extract.<locals>.<lambda> at 0x10ca599d8>, 'get_output_feature_dim': <function Conv2DFrontExtractor.extract.<locals>.<lambda> at 0x10ca59a60>, 'get_weights_permute': Permutation(perm=array([3, 2, 0, 1]), inv=array([2, 3, 1, 0])), 'dim_attrs': ['spatial_dims', 'channel_dims', 'axis', 'batch_dims'], 'shape_attrs': ['output_shape', 'stride', 'window', 'shape', 'pad'], 'IE': [('layer', [('id', <function Op.substitute_ie_attrs.<locals>.<lambda> at 0x10ca5a730>), 'name', 'precision', 'type'], [('data', ['auto_pad', 'group', ('strides', <function Convolution.backend_attrs.<locals>.<lambda> at 0x10ca5a7b8>), ('dilations', <function Convolution.backend_attrs.<locals>.<lambda> at 0x10ca5a840>), ('kernel', <function Convolution.backend_attrs.<locals>.<lambda> at 0x10ca5a8c8>), ('pads_begin', <function Convolution.backend_attrs.<locals>.<lambda> at 0x10ca5a950>), ('pads_end', <function Convolution.backend_attrs.<locals>.<lambda> at 0x10ca5a9d8>), 'output', 'pad_value', 'mode', 'input'], []), '@ports', '@consts'])], 'is_output_reachable': True, 'is_undead': False, 'is_const_producer': False, 'is_partial_inferred': False, 'kernel_shape': array([ 3, 3, 3, 32]), 'kernel_spatial_idx': array([0, 1]), 'spatial_dims': array([], dtype=float64), 'kernel_spatial': array([3, 3]), 'output': 32, 'pad': array([], dtype=int64)}

E0725 16:47:54.102306 4391695808 main.py:307] Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "Conv1_layer/Conv" node.

For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

I0725 16:47:54.111489 4391695808 main.py:308] Traceback (most recent call last):

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 132, in partial_infer

node.infer(node)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/ops/convolution.py", line 187, in infer

node['pad_spatial_shape'] = node.pad[node.spatial_dims]

IndexError: arrays used as indices must be of integer (or boolean) type

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 273, in apply_replacements

for_graph_and_each_sub_graph_recursively(graph, replacer.find_and_replace_pattern)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/middle/pattern_match.py", line 58, in for_graph_and_each_sub_graph_recursively

func(graph)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/extensions/middle/PartialInfer.py", line 31, in find_and_replace_pattern

partial_infer(graph)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/middle/passes/infer.py", line 198, in partial_infer

refer_to_faq_msg(38)) from err

mo.utils.error.Error: Stopped shape/value propagation at "Conv1_layer/Conv" node.

For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/main.py", line 302, in main

return driver(argv)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/main.py", line 251, in driver

is_binary=not argv.input_model_is_text)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/pipeline/tf.py", line 134, in tf2nx

class_registration.apply_replacements(graph, class_registration.ClassType.MIDDLE_REPLACER)

File "/opt/intel/openvino_2019.2.242/deployment_tools/model_optimizer/mo/utils/class_registration.py", line 293, in apply_replacements

)) from err

mo.utils.error.Error: Exception occurred during running replacer "REPLACEMENT_ID" (<class 'extensions.middle.PartialInfer.PartialInfer'>): Stopped shape/value propagation at "Conv1_layer/Conv" node.

For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #38.

Shubha_R_Intel · ‎07-25-2019

Dear Friedman, Benny, I tried OpenVino 2019R2 against the model you attached to your previous post and I got the same error you did. I printed your frozen pb file into a pbtxt and saw that layer "input" is the input into layer "Conv1_layer/Conv." But when you go back and look at "input" you see this (look at the shape being unknown_rank in the snippet below). If you pass in an actual --input_shape to your mo_tf.py command the Model Optimizer should be able to infer the shape for the Conv1_layer/Conv layer.

[1,227,227,3] is the format of what you should pass along with the --input_shape, where 1 is batch_size, 227 is Height, 227 is Width and 3 is Number_of_Channels or NHWC since this is Tensorflow Layout (of course your N, H and W may be different)

Model Optimizer actually tells you the exact reason in its error:

E0725 12:04:58.149042 10916 infer.py:183] It can happen due to bug in custom shape infer function <function Convolution.infer at 0x00000201C37A0510>.
E0725 12:04:58.149042 10916 infer.py:184] Or because the node inputs have incorrect values/shapes.
E0725 12:04:58.164666 10916 infer.py:185] Or because input shapes are incorrect (embedded to the model or passed via --input_shape).
I0725 12:04:58.164666 10916 infer.py:196] Node "Conv1_layer/Conv" attributes: {'pb': name: "Conv1_layer/Conv"

Snippet from the pbtxt version of your frozen.pb :

node {
name: "prefix/input"
op: "Placeholder"
attr {
key: "dtype"
value {
type: DT_FLOAT
}
}
attr {
key: "shape"
value {
shape {
unknown_rank: true
}
}
}
}

Hope it helps and thanks for upgrading to R2 !

Shubha

Friedman__Benny · ‎07-29-2019

Shubha,

This was very helpful. I added data for all three input parameters:

python3 /opt/intel/openvino_2019.1.144/deployment_tools/model_optimizer/mo.py --input input,drop_rate,skip_drop --input_shape [20,150,150,3],[1],[1] --input_model cnn.pb

However, I now get a new error which I'm unable to resolve:

E0729 21:23:06.177761 4684252608 main.py:317] Can't permute attrs for node EltwiseReshapeNormalization. Error message: only integer scalar arrays can be converted to a scalar index

Any chance of getting help on this?

benny

Shubha_R_Intel · ‎07-31-2019

Dear Friedman, Benny,

The below looks like it's coming from numpy, not from Model Optimizer.

Can't permute attrs for node EltwiseReshapeNormalization. Error message: only integer scalar arrays can be converted to a scalar index

Please look at the following Stack Overflow link:

https://stackoverflow.com/questions/50997928/typeerror-only-integer-scalar-arrays-can-be-converted-to-a-scalar-index-with-1d

Model Optimizer is simply passing the error along.

Thanks,

Shubha

Friedman__Benny · ‎08-01-2019

Shubha,

Thank you for this answer. I have mistakenly ran the older mo.py. on 2019R2 this does not reproduce.

I do get the unsupported ops error:

E0801 15:21:56.816384 4645127616 unsupported_ops.py:38] List of operations that cannot be converted to Inference Engine IR:

E0801 15:21:56.816710 4645127616 unsupported_ops.py:40] RandomUniform (1)

E0801 15:21:56.816910 4645127616 unsupported_ops.py:42] Dropout_layer/RandomUniform

E0801 15:21:56.817031 4645127616 unsupported_ops.py:40] Floor (1)

E0801 15:21:56.817157 4645127616 unsupported_ops.py:42] Dropout_layer/Floor

E0801 15:21:56.817305 4645127616 main.py:307] Part of the nodes was not converted to IR. Stopped.

For more information please refer to Model Optimizer FAQ (https://docs.openvinotoolkit.org/latest/_docs_MO_DG_prepare_model_Model_Optimizer_FAQ.html), question #24.

Which is kind of surprising as RandomUniform and Floor are basic primitives.

benny

Shubha_R_Intel · ‎08-01-2019

Dear Friedman, Benny,

Yep. According to Supported Framework Layers Doc RandomUniform is mentioned as not supported and Floor is just not mentioned at all (which means it's not supported).

You can build your own custom layer to support them though. All is not lost. This is a slightly old post which talks about ONNX but the concepts are still relevant here:

Custom Layers IDZ Post

Many customers have developed custom layers - it is not too difficult.

Maybe give it a try ?

Thanks,

Shubha