IndexError: list index out of range trying to compile TF model

[idata]

Traceback (most recent call last):

File "/usr/local/bin/mvNCCompile", line 118, in create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights) File "/usr/local/bin/mvNCCompile", line 104, in create_graph net = parse_tensor(args, myriad_config, debug=True) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 1007, in parse_tensor output_size = [input_shape[0], 1, 1, outputs[3]] File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 521, in getitem return self._dims[key]IndexError: list index out of range

This occurs if I pass either my TF meta or pb file like so:

mvNCCompile model.pb -w incept3_ft_weights.h5 -s 12 -in inception_v3_input -on dense_25/Softmax -o graph

What am I doing wrong?

[idata]

Hi @Tome_at_Intel ,

We got the similar problem, the log message as the following:

Could you help suggest us how to look into & fix the issue? Thanks.BR.

(tensorflow) ubuntux64@ubuntux64-ThinkPad-X231s:~/workspace/cdc/test_create_graph$ mvNCCompile model_inference.meta -w model_inference -s 12 -in images -on predictions -o cdc_inference.graph

mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py:858: DeprecationWarning: builtin type EagerTensor has no module attribute

EagerTensor = c_api.TFE_Py_InitEagerTensor(_EagerTensorBase)/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead if d.decorator_argspec is not None), _inspect.getargspec(target))/usr/local/lib/python3.5/dist-packages/h5py/init.py:34: FutureWarning: Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).type. from ._conv import register_converters as _register_converters/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead if d.decorator_argspec is not None), _inspect.getargspec(target))/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead if d.decorator_argspec is not None), _inspect.getargspec(target))/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py:539: DeprecationWarning: The binary mode of fromstring is deprecated, as it behaves surprisingly on unicode inputs. Use frombuffer instead return np.fromstring(tensor.tensor_content, dtype=dtype).reshape(shape)Fusing depthconv and conv in model/block2_sepconv1/separable_conv2d/depthwise and model/block2_sepconv1/separable_conv2dFusing depthconv and conv in model/block2_sepconv2/separable_conv2d/depthwise and model/block2_sepconv2/separable_conv2dFusing depthconv and conv in model/block3_sepconv1/separable_conv2d/depthwise and model/block3_sepconv1/separable_conv2dFusing depthconv and conv in model/block3_sepconv2/separable_conv2d/depthwise and model/block3_sepconv2/separable_conv2dFusing depthconv and conv in model/block4_sepconv1/separable_conv2d/depthwise and model/block4_sepconv1/separable_conv2dFusing depthconv and conv in model/block4_sepconv2/separable_conv2d/depthwise and model/block4_sepconv2/separable_conv2dFusing depthconv and conv in model/block5_sepconv1/separable_conv2d/depthwise and model/block5_sepconv1/separable_conv2dFusing depthconv and conv in model/block5_sepconv2/separable_conv2d/depthwise and model/block5_sepconv2/separable_conv2dFusing depthconv and conv in model/block5_sepconv3/separable_conv2d/depthwise and model/block5_sepconv3/separable_conv2dFusing depthconv and conv in model/block6_sepconv1/separable_conv2d/depthwise and model/block6_sepconv1/separable_conv2dFusing depthconv and conv in model/block6_sepconv2/separable_conv2d/depthwise and model/block6_sepconv2/separable_conv2dFusing depthconv and conv in model/block6_sepconv3/separable_conv2d/depthwise and model/block6_sepconv3/separable_conv2dFusing depthconv and conv in model/block7_sepconv1/separable_conv2d/depthwise and model/block7_sepconv1/separable_conv2dFusing depthconv and conv in model/block7_sepconv2/separable_conv2d/depthwise and model/block7_sepconv2/separable_conv2dFusing depthconv and conv in model/block7_sepconv3/separable_conv2d/depthwise and model/block7_sepconv3/separable_conv2dFusing depthconv and conv in model/block8_sepconv1/separable_conv2d/depthwise and model/block8_sepconv1/separable_conv2dFusing depthconv and conv in model/block8_sepconv2/separable_conv2d/depthwise and model/block8_sepconv2/separable_conv2dFusing depthconv and conv in model/block8_sepconv3/separable_conv2d/depthwise and model/block8_sepconv3/separable_conv2dFusing depthconv and conv in model/block13_sepconv1/separable_conv2d/depthwise and model/block13_sepconv1/separable_conv2dFusing depthconv and conv in model/block13_sepconv2/separable_conv2d/depthwise and model/block13_sepconv2/separable_conv2dFusing depthconv and conv in model/block14_sepconv1/separable_conv2d/depthwise and model/block14_sepconv1/separable_conv2dFusing depthconv and conv in model/block14_sepconv2/separable_conv2d/depthwise and model/block14_sepconv2/separable_conv2dTraceback (most recent call last): File "/usr/local/bin/mvNCCompile", line 118, in create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights) File "/usr/local/bin/mvNCCompile", line 104, in create_graph net = parse_tensor(args, myriad_config) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 1007, in parse_tensor print('outputs[3] : %s',outputs[3])#print(input_shape[0]) ->±¾¦b³o¦æ(¦]¬°¥h¦s¨úoutputs[3]³o­Ó¤¸¯À,·|¥hprint³o­Ó¬O¦]¬°³o¦æ:output_size = [input_shape[0], 1, 1, outputs[3]] File "/home/ubuntux64/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 521, in getitem return self._dims[key]IndexError: list index out of range

[idata]

@samhsieh Which model is this? Can you provide a link to the model for reproduction purposes?

[idata]

@Tome_at_Intel,

Thanks for your feedback.

We pre-trained the model with Tensowflow_GPU (v1.6) in Ubuntu 16.04,and input layer name: "images", output layer name: "predictions".Then run the following command to generate "cdc_inference.graph": mvNCCompile model_inference.meta -w model_inference -s 12 -in images -on predictions -o cdc_inference.graphHere is the link https://drive.google.com/drive/folders/16WLC8VwLrGTGtWAGu04DGdJIIqEFHqdH

[idata]

@samhsieh For this network, the list index out of range error seems to be occurring during the mean operation. Specifically during line 1028 of the TensorFlowParser.py file, during which the output shape is being set: output_size = [input_shape[0], 1, 1, outputs[3]]. Can you double check your TensorFlow code and see if there could be any issues that could cause this statement to not execute properly?

[idata]

@Tome_at_Intel , we observed that there is only one element in output shape of "mean" layer operation node after Enable "debug" flag(TRUE) in TensorFlowParser.py.

BTW, even though we modified the outputs[0], it still got the failure:tensor_shape.py file, line 582, in merge_with ValueError:Shapes (?, 2048) and (?, 1, 1,?) are not compatible.Why does the parser check the specified component of output?We would appreciate it that you give us some suggestions.BR,

[idata]

@Tome_at_Intel, @samhsieh Any update how on how to fix it?

[idata]

I don't have a specific update to this, but please try your network with NCSDK 2.08 and let me know if the issue persists.

[idata]

I'm facing a similar error

_* Info: No Weights provided. inferred path: TF_Model/tf_model.data-00000-of-00001_*

shape: [1, 224, 224, 3][Error 5] Toolkit Error: Stage Details Not Supported: FusedBatchNorm inputs mean and variance are not defined. The graph is not created for inference

I was facing the Index error message earlier on NCSDK v1.12. But when updated it to v.2.08.01.02 , I started facing the above error

Meta file at the below location

https://drive.google.com/drive/folders/12ZwSaxyEjPg7OAxzSWgz0OXpkLkErDlG?usp=sharing

Can someone please check ??? Any updates/WA would be appreciated!!

[idata]

I came here to complain about the Batch Normalization issue as well, but as I was copying my code I noticed my mistake.

Here's the offending line:

net = tf.layers.batch_normalization(net, training=training, name="bn_{}".format(i + 1))

I had forgotten about the "training" parameter, and it was set to true. I don't know what it does or what it's even meant for, but when I set it to false, my graph compiled with the latest NCSDK just fine

EDIT:

Scratch that, changing "training" to false just breaks my network, and makes my output look green… Still looking for a solution