KeyError: "The name 'input:0' refers to a Tensor which does not exist." when compile retrained model

[idata]

Hello, guys. Now I retrained an Inception-v3 model to fine-tune it's output layer to 11 classes. Originally it may output 1001 classes but in our case we only need to calssify 11 classes so we retrained it.It works well but when I tried to compile it into graph some error happens and please help me.

For this model, I have a frozen .pb file and some ckpt files like model.ckpt.meta or model.ckpt.index and so on.

And in terminal, I use the command:(tensorflow) wxy@wxy-mipro:~/Documents/TensorFLow/retrained/ckpt$ sudo mvNCCompile model.ckpt.meta -in=input -is 299 299 -o inception-V3-retrained.graph

and it outputs:

mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

/usr/local/lib/python3.6/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py:766: DeprecationWarning: builtin type EagerTensor has no module attribute EagerTensor = c_api.TFE_Py_InitEagerTensor(_EagerTensorBase)/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.5 of module 'tensorflow.python.framework.fast_tensor_util' does not match runtime version 3.6 return f(*args, **kwds)/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() or inspect.getfullargspec() if d.decorator_argspec is not None), _inspect.getargspec(target))Traceback (most recent call last): File "/usr/local/bin/mvNCCompile", line 118, in create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights) File "/usr/local/bin/mvNCCompile", line 104, in create_graph net = parse_tensor(args, myriad_config) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 237, in parse_tensor inputTensor = graph.get_tensor_by_name(inputnode + ':0') File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3207, in get_tensor_by_name return self.as_graph_element(name, allow_tensor=True, allow_operation=False) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3035, in as_graph_element return self._as_graph_element_locked(obj, allow_tensor, allow_operation) File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/ops.py", line 3077, in _as_graph_element_locked "graph." % (repr(name), repr(op_name)))KeyError: "The name 'input:0' refers to a Tensor which does not exist. The operation, 'input', does not exist in the graph."

[idata]

@Tome_at_Intel the file is just still the same as in this link: https://drive.google.com/open?id=125pP5Nkmqf1eBxZnVfdMizAMfaKM2N4n

[idata]

@Tome_at_Intel Hello? Can you give me some help? The file is just the same file, but I can not get meaningful compiled file, although there is no error message at all.

[idata]

@WuXinyang I'll take a look at it WuXinYang and let you know what I find. Thank you for your patience.

[idata]

@WuXinyang I have no issues with using mvNCCompile and mvNCCheck with this model. Do you have a python program that you are using for this model? If you do, I'd like to try it out. My results are below using mvNCCheck output_graph.pb -in input/BottleneckInputPlaceholder -on final_result -s 12

USB: Transferring Data... USB: Myriad Execution Finished USB: Myriad Connection Closing. USB: Myriad Connection Closed. Result: (1, 1, 11) 1) 8 0.5576 2) 10 0.0822 3) 4 0.0806 4) 7 0.0781 5) 6 0.05487 Expected: (1, 11) 1) 8 0.5606347 2) 10 0.081698544 3) 4 0.08018297 4) 7 0.07761778 5) 6 0.05446419 ------------------------------------------------------------ Obtained values ------------------------------------------------------------ Obtained Min Pixel Accuracy: 0.5382266361266375% (max allowed=2%), Pass Obtained Average Pixel Accuracy: 0.0993441790342331% (max allowed=1%), Pass Obtained Percentage of wrong values: 0.0% (max allowed=0%), Pass Obtained Pixel-wise L2 error: 0.17263926923498502% (max allowed=1%), Pass Obtained Global Sum Difference: 0.00612653698772192 ------------------------------------------------------------

[idata]

@Tome_at_Intel Thanks for your reply! I can also use my mvNCCheck get the same results as you. And I can also load the outputted graph into NCS but can not get right predictions.

I uploaded the code I use in Google Drive: https://drive.google.com/open?id=1oqscNSlCIBHTc1ijXkxHHDeI_LJoaK5S

Originally, without NCS, I use label_image.py to get right predictions with original output_graph.pb file, like:

$ python3 label_image.py test1.jpgtest1.jpgbanana (score = 0.99928)pear (score = 0.00045)apple (score = 0.00019)melon (score = 0.00003)peach (score = 0.00002)carrot (score = 0.00001)lettuce (score = 0.00001)berry (score = 0.00001)onion (score = 0.00000)cabbage (score = 0.00000)tomato (score = 0.00000)

But with NCS, by using run_retrained_1.py, I cannot get right predictions with compiled graph, like:

$ python3 run_retrained_1.py test1.jpgNumber of categories: 11Start download to NCS…

inception-v3 on NCS

8 carrot 1.0

10 melon 0.09 banana 0.07 apple 0.06 onion 0.0

Finished

The right prediction for test1.jpg shall be banana, but NCS only outputs carrot. In fact, no matter what images, it will only output carrot.

Maybe I did not write the script right. Thanks for your kind reply again and look forward for your instructions!

[idata]

@WuXinyang Thanks for providing your files. I am able to reproduce this issue as well. I think it may be a preprocessing issue. I'll keep looking into this.

[idata]

@Tome_at_Intel Really thanks for your kind reply!!

[idata]

@WuXinyang Thanks for providing your files. I am able to reproduce this issue as well. I tried a few image preprocessing steps (whiten image, color channel transposition, checked the input resize dimensions, mean subtraction, etc.) and still running into this issue. Not sure why this is happening but I haven't seen this kind of network before. mvNCProfile shows only one layer and you're correct about the graph size being incredibly small.

[idata]

@Tome_at_Intel Hi Tome, may I ask that if there are any other new topics which have similar problems like me? I still have not resolved this problem :(

[idata]

Hello, @Tome_at_Intel. I got the same problem that I don't know the name of the input layer. Would you please tell me how to use the Tensorflow's summarize graph tool? Thank you very much.

[idata]

@tenngre Please visit https://github.com/tensorflow/tensorflow/tree/master/tensorflow/tools/graph_transforms and clone the tensorflow repo. Install Bazel by following the directions at https://docs.bazel.build/versions/master/install-ubuntu.html. You can then run the commands: bazel build tensorflow/tools/graph_transforms:summarize_graph and then bazel-bin/tensorflow/tools/graph_transforms/summarize_graph --in_graph=tensorflow_inception_graph.pb to inspect your model from inside of the cloned TensorFlow directory.

Please visit the "Inspecting Graphs" section of TensorFlow github page for more information.

[idata]

@Tome_at_Intel thank you for your kind prompt reply. I will try this and tell you the result, thank you very much.

[idata]

@Tome_at_Intel , I am sorry to bother again. I followed your answer to install the Bazel by brew in my Macbook pro. I also clone the tensorflow repo (I install the tensorflow (version 1.8.0) by anaconda in a virtual environment, does this cause the following error?), however, the commands: bazel build tensorflow/tools/graph_transforms:summarize_graph cannot run on my pc. After I input the command, the error shows:

ERROR: Skipping 'tensorflow/tools/graph_transforms:summarize_graph': no such package 'tensorflow/tools/graph_transforms/tensorflow/tools/graph_transforms': BUILD file not found on package path WARNING: Target pattern parsing failed. ERROR: no such package 'tensorflow/tools/graph_transforms/tensorflow/tools/graph_transforms': BUILD file not found on package path INFO: Elapsed time: 0.605s INFO: 0 processes. FAILED: Build did NOT complete successfully (0 packages loaded)

Would you please tell me what has happened and the solution?

[idata]

@Tome_at_Intel The above problem is solved.

command in github is bazel build tensorflow/tools/graph_transforms:summarize_graph.I tried to use bazel build //tensorflow/tools/graph_transforms:summarize_graph. it works.

I checked the .pd file of the _Pre-trained Models_ of MobileNet_v1_1.0_224, which is downloaded from

Github. The result isFound 1 possible inputs: (name=input, type=float(1), shape=[?,224,224,3]) No variables spotted. Found 1 possible outputs: (name=MobilenetV1/Predictions/Reshape_1, op=Reshape) Found 4254891 (4.25M) const parameters, 0 (0) variable parameters, and 0 control_edges Op types used: 138 Identity, 138 Const, 27 Relu6, 27 FusedBatchNorm, 15 Conv2D, 13 DepthwiseConv2dNative, 2 Reshape, 1 Placeholder, 1 AvgPool, 1 BiasAdd, 1 Shape, 1 Softmax, 1 Squeeze To use with tensorflow/tools/benchmark:benchmark_model try these arguments: bazel run tensorflow/tools/benchmark:benchmark_model -- --graph=mobilenet_v1_1.0_224_frozen.pb --show_flops --input_layer=input --input_layer_type=float --input_layer_shape=-1,224,224,3 --output_layer=MobilenetV1/Predictions/Reshape_1

So, I am sure that the name of the input layer is _input_. However, when I try to convert this model to a graph file by

1. mvNCCheck mobilenet_v1_1.0_224.ckpt.meta -in=input -on=MobilenetV1/Predictions/Reshape_1 -s 12 -of mobilenet.graph
2. mvNCCompile mobilenet_v1_1.0_224.ckpt.meta -in=input -on=MobilenetV1/Predictions/Reshape_1 -s 12 -o mobilenet.graph
3. mvNCProfile mobilenet_v1_1.0_224.ckpt.meta -in=input -on=MobilenetV1/Predictions/Reshape_1 -s 12 -o mobilenet.graph

*mvNCCheck v02.00, Copyright @ Movidius Ltd 2016* /usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py:766: DeprecationWarning: builtin type EagerTensor has no __module__ attribute EagerTensor = c_api.TFE_Py_InitEagerTensor(_EagerTensorBase) /usr/local/lib/python3.5/dist-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead if d.decorator_argspec is not None), _inspect.getargspec(target)) Traceback (most recent call last): File "/usr/local/bin/mvNCCheck", line 152, in <module> quit_code = check_net(args.network, args.image, args.inputnode, args.outputnode, args.nshaves, args.inputsize, args.weights, args) File "/usr/local/bin/mvNCCheck", line 130, in check_net net = parse_tensor(args, myriad_config, file_gen=True) File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 227, in parse_tensor inputTensor = graph.get_tensor_by_name(inputnode + ':0') File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3207, in get_tensor_by_name return self.as_graph_element(name, allow_tensor=True, allow_operation=False) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3035, in as_graph_element return self._as_graph_element_locked(obj, allow_tensor, allow_operation) File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3077, in _as_graph_element_locked "graph." % (repr(name), repr(op_name))) KeyError: "The name 'input:0' refers to a Tensor which does not exist. The operation, 'input', does not exist in the graph."

The mvNC* commands seem to say the name of the input layer is wrong. What should I do next? Thank you.

[idata]

@tenngre It seems like the mobilenet you are using has an undefined shape in the input, but I don't know if this would cause the NCSDK to not find the input node. The NCSDK does expect all input shape dimensions to be defined explicitly.

The mobilnet model from the link you provided also came with a mobilenet pb file which I had no issues compiling and checking using the same input and output nodes (input, MobilenetV1/Predictions/Reshape_1). Maybe you can use this model instead of the .meta version?

We also have a variety of mobilenet model samples available in the ncappzoo if you are interested.

[idata]

@Tome_at_Intel Hi, recently I got some progress.

With the tensorflow's graph_transform tools, I converted the original retrained frozen .pb file into .pbtxt file, which is not in binary but in txt, so I can easily check the names of the nodes in the graph. And I found that the right input node of the graph shall be DecodeJpeg/contents, which is also in fact the input node name I used for my own inferrence script without NCS, you can check that in the scripts I ever uploaded. And then I tried to mvNCCompile with the DecodeJpeg/contents, and then it returned error 5 saying that there is a error in TensorSparse.py.I debugged the error and found it is because of the shape . In TensorParse.py, the shape of the input node DecodeJpeg/contents is (), although in fact I already specify the -is as 299 299.And also about the DecodeJpeg/contents node, I checked in the .pbtxt file, it contains two attributes, one attribute is "dtype", another attribute is "value". And for the "value" attribute, it has attribute "tensorshape" , which is just the default shape of Inception-V3 midel: 299 299.So the problem seems that in your SDK, the TensorParse.py can not right extract the node's shape.The model I talked above is the myself retrained model. To check and compare the difference, I downloaded a frozen Inception-v3 model from tensorflow and also converted it into .pbtxt. And by comparision, the Input node of the Inception-v3 model(the one I downloaded from tensorflow, not retrained one!) is exactly the same as my retrained model. In fact, the only difference between the original model and my retrained model is that some weights changed, and there are some more layers called "final_result*" in the end of the retrained .pbtxt file.It makes sense. Since the transfer learning means just only modify the weights and output layers and keep all the other structure the same.If you are still interested I can upload more files and updated some information.