Tensorflow -> mvNCCompile issue

[idata]

I'm trying to compile a custom Tensorflow model using mvNCCompile, and am running into an issue that I can't figure out. I've isolated it to a very simple function where I am doing a 2D upsample using only concatenations and reshapes. It works fine when running in Tensorflow, but I get a strange error when compiling. I think it has something to do with the way mvNCCompile handles concatenation. Any help is greatly appreciated.

Code:

import tensorflow as tf def upsample(x): shape = x.get_shape().as_list() r1 = tf.reshape(x, [shape[0], shape[1] * shape[2], 1, shape[3]]) r2 = tf.concat([r1, r1], axis=2) r3 = tf.reshape(r2, [shape[0], shape[1], shape[2] * 2, shape[3]]) r4 = tf.concat([r3, r3], axis=2) r5 = tf.reshape(r4, [shape[0], shape[1] * 2, shape[2] * 2, shape[3]]) return r5 sess = tf.Session() input_placeholder = tf.placeholder(tf.float32, [1, 16, 16, 3]) output = upsample(input_placeholder) graph = sess.graph_def with open("test.pb", "wb") as f: f.write(graph.SerializeToString()) print(input_placeholder.name) print(output.name)

Error:

mvNCCompile /media/sf_vb_shared/test.pb -in import/Placeholder -on import/Reshape_2Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 563, in merge_with new_dims.append(dim.merge_with(other[i])) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 138, in merge_with self.assert_is_compatible_with(other) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 111, in assert_is_compatible_with other)) ValueError: Dimensions 2 and 1 are not compatible During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/bin/mvNCCompile", line 118, in <module> create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights) File "/usr/local/bin/mvNCCompile", line 104, in create_graph net = parse_tensor(args, myriad_config) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 934, in parse_tensor [a_input[0], a_input[1], a_input[2], concat_channel_size]) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 378, in set_shape self._shape = self._shape.merge_with(shape) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 566, in merge_with raise ValueError("Shapes %s and %s are not compatible" % (self, other)) ValueError: Shapes (1, 256, 2, 3) and (1, 256, 1, 6) are not compatible

[idata]

Looking at the code, is it the case that only channel-wise concatenation is supported?

[idata]

A couple of things I've noticed while converting a relatively complex Tensorflow model to be Movidius compatible:

• elu activation is listed as supported in the ncsdk releases, but it is not (at least for TF)
• there really isn't any way to upsample layers that I am aware of (except for the extremely hacky below solution)
• seems like placeholder_with_default is not supported
• I couldn't get passing in input shape to work correctly, so I had to hard-code it into my model
• I had to convert an add_n of 3 layers to 2 separate addition ops
• error messages are extremely cryptic, for example when an op is not supported, would be great to get the tensor name instead of just the op name
• there is a bug in TensorFlowParser.py, padding_tracker is not checked when looking for an op's input (in function get_input), so it can't find inputs for an op if its input is a padding op
• when there aren't enough resources to fit the model on the Myriad chip, it would be great to get some kind of indication of how close the model is to fitting ie something like: 100MB max, your model is 104MB

In the end I got it working though.

A Movidius compatible nearest-neighbor upsampling in Tensorflow (pretty hacky, but it works):

def upsample(self, x): shape = x.get_shape().as_list() r1 = tf.reshape(x, [shape[0], shape[1] * shape[2], 1, shape[3]]) r1_l = tf.pad(r1, [[0, 0], [0, 0], [0, 1], [0, 0]]) r1_r = tf.pad(r1, [[0, 0], [0, 0], [1, 0], [0, 0]]) r2 = tf.add(r1_l, r1_r) r3 = tf.reshape(r2, [shape[0], shape[1], shape[2] * 2, shape[3]]) r3_l = tf.pad(r3, [[0, 0], [0, 0], [0, shape[2] * 2], [0, 0]]) r3_r = tf.pad(r3, [[0, 0], [0, 0], [shape[2] * 2, 0], [0, 0]]) r4 = tf.add(r3_l, r3_r) r5 = tf.reshape(r4, [shape[0], shape[1] * 2, shape[2] * 2, shape[3]]) return r5

[idata]

@tripleplay369 Thanks for all of your feed back and for bringing the concat issue to our attention. I'm glad that you've been able to find a solution that works for you. Regarding the Tensorflow concat error, we've reproduced the issue and we're working on a fix. Thanks again!

[idata]

My network uses upsampling and i get here. I tried @tripleplay369's solution, and add one conv layer after upsample, the code is as follows:

import tensorflow as tf from tensorflow.python.framework import graph_util from tensorflow.python.framework import graph_io def upsample(x): shape = x.get_shape().as_list() print("x shape: {}".format(x.get_shape().as_list())) r1 = tf.reshape(x, [shape[0], shape[1] * shape[2], 1, shape[3]]) print("r1 shape: {}".format(r1.get_shape().as_list())) r1_l = tf.pad(r1, [[0, 0], [0, 0], [0, 1], [0, 0]]) print("r1_l shape: {}".format(r1_l.get_shape().as_list())) r1_r = tf.pad(r1, [[0, 0], [0, 0], [1, 0], [0, 0]]) print("r1_r shape: {}".format(r1_r.get_shape().as_list())) r2 = tf.add(r1_l, r1_r) r3 = tf.reshape(r2, [shape[0], shape[1], shape[2] * 2, shape[3]]) r3_l = tf.pad(r3, [[0, 0], [0, 0], [0, shape[2] * 2], [0, 0]]) r3_r = tf.pad(r3, [[0, 0], [0, 0], [shape[2] * 2, 0], [0, 0]]) r4 = tf.add(r3_l, r3_r) r5 = tf.reshape(r4, [shape[0], shape[1] * 2, shape[2] * 2, shape[3]]) return r5 sess = tf.Session() x = tf.placeholder(tf.float32, [1, 16, 16, 3], name="input") output = upsample(x) output = tf.layers.conv2d(output, 32, 3, 1, padding="same") output = tf.identity(output, "output") init_op = tf.global_variables_initializer() sess.run(init_op) constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), ["output"]) graph_io.write_graph(constant_graph, ".", "test.pb", as_text=False) print(x.name) print(x.get_shape()) print(output.name) print(output.get_shape())

this script will save a test.pb, and then i mvNCCompiled it and got this error:

Stage Details Not Supported: Top Not Found Add_1

I looked into TensorflowParser.py in ncsdk (version 2.05), and i think there is some oversight from line 1146:

elif ((node.type == 'Add' or node.type == 'Mul' or node.type == 'Maximum') and get_input(strip_tensor_id(node.inputs[0].name), False) != 0 and get_input(strip_tensor_id(node.inputs[1].name), False) != 0 and (prev_node is None or (prev_node.op != StageType.convolution and prev_node.op != StageType.fully_connected_layer and prev_node.op != StageType.depthwise_convolution) or prev_node.bias is not None)):

If top of node 'Add' is a pad node, then it will miss this elif judge. we should check both get_padding_input and get_input to get top

node.

If i am correct, how to fix this error? Or are there some other ways to support upsample?

[idata]

@huchhong Hi, I got the solution for "Stage Details Not Supported: Top Not Found Add_1", (although maybe it's too late, hhhh) thanks to the explanation from @tripleplay369 , just add a padding_tracker check in get_input (in TensorFlowParser.py)like this:

line 103 global padding_tracker if len(padding_tracker) != 0: ----for padding in padding_tracker: --------if padding[0] == name: -------------return get_input(padding[1])The error raised because layer add_1 can't find its top node - pad. Because in the func "get_input", there's no padding_tracker check, pads are ignored.And for the bug mentioned by tripleplay369: " I had to convert an add_n of 3 layers to 2 separate addition ops", he solved it by put two pads instead one at a time.