Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

mvNCCompile has problem with Dropout

idata
Employee
825 Views

Hi,

 

I have a Tensorflow model saved, and I am trying to use

 

compilable frozen_model.pb -in conv_I_input -on dense_6/Softmax -o output

 

But I am getting the error below. Any idea how can I get rid of it?

 

Note: the model was originally defined by keras, but saved as frozen Tensorflow model!

 

Thanx in advance

 

Sandor

 

 

mvNCCompile v02.00, Copyright @ Movidius Ltd 2016

 

/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py:766: DeprecationWarning: builtin type EagerTensor has no module attribute

 

EagerTensor = c_api.TFE_Py_InitEagerTensor(_EagerTensorBase)

 

/usr/local/lib/python3.5/dist-packages/tensorflow/python/util/tf_inspect.py:45: DeprecationWarning: inspect.getargspec() is deprecated, use inspect.signature() instead

 

if d.decorator_argspec is not None), _inspect.getargspec(target))

 

Traceback (most recent call last):

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1323, in _do_call

 

return fn(*args)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1302, in _run_fn

 

status, run_metadata)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors_impl.py", line 473, in exit

 

c_api.TF_GetCode(self.status.status))

 

tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'dropout_5/keras_learning_phase' with dtype bool

 

[[Node: dropout_5/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):

 

File "/usr/local/bin/mvNCCompile", line 118, in

 

create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights)

 

File "/usr/local/bin/mvNCCompile", line 104, in create_graph

 

net = parse_tensor(args, myriad_config)

 

File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 276, in parse_tensor

 

res = outputTensor.eval(feed_dict={inputnode + ':0' : input_data})

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 570, in eval

 

return _eval_using_default_session(self, feed_dict, self.graph, session)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 4455, in _eval_using_default_session

 

return session.run(tensors, feed_dict)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 889, in run

 

run_metadata_ptr)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1120, in _run

 

feed_dict_tensor, options, run_metadata)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1317, in _do_run

 

options, run_metadata)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 1336, in _do_call

 

raise type(e)(node_def, op, message)

 

tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'dropout_5/keras_learning_phase' with dtype bool

 

[[Node: dropout_5/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

 

Caused by op 'dropout_5/keras_learning_phase', defined at:

 

File "/usr/local/bin/mvNCCompile", line 118, in

 

create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights)

 

File "/usr/local/bin/mvNCCompile", line 104, in create_graph

 

net = parse_tensor(args, myriad_config)

 

File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 211, in parse_tensor

 

tf.import_graph_def(graph_def, name="")

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/importer.py", line 313, in import_graph_def

 

op_def=op_def)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2956, in create_op

 

op_def=op_def)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1470, in init

 

self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

 

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'dropout_5/keras_learning_phase' with dtype bool

 

[[Node: dropout_5/keras_learning_phase = Placeholder[dtype=DT_BOOL, shape=, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
0 Kudos
4 Replies
idata
Employee
495 Views

Hi,

 

It turned out that the problem is not only related to Dropout….

 

I seen

 

https://github.com/ardamavi/Intel-Movidius-NCS-Keras

 

where the same InvalidArgumentError was seen on the input layer…..

 

Anyone else seen this error?

 

Thanx

 

S&|
0 Kudos
idata
Employee
495 Views

@sseres, The key to you issue is in this error statement:

 

 

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'dropout_5/keras_learning_phase' with dtype bool

 

 

Looks like your frozen model still has variables in it, which shouldn't be the case when performing inference on any mobile/embedded device. Here's a simple example network that uses dropout layer - https://github.com/ashwinvijayakumar/ncappzoo/tree/single-conv/tensorflow/single-conv.

 

 

At the time of writing this response, I haven't yet merged my image-classifier updates, so replace apps/image-classifier with https://github.com/ashwinvijayakumar/ncappzoo/tree/classifier-template/apps/image-classifier before following instructions on tensorflow/single-conv/README.md

 

0 Kudos
idata
Employee
495 Views

Hi @AshwinVijayakumar,

 

Thanx for the response.

 

This is how I saved my model:

 

K.set_learning_phase(0)

 

K._LEARNING_PHASE = tf.constant(0)

 

file_name = 'orig_graph.pb'

 

saver = tf.train.Saver(tf.global_variables())

 

with tf.Session() as sess:

 

sess.run(tf.global_variables_initializer())

 

sess.run(tf.local_variables_initializer())

 

constant_graph = graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), ['dense_6/Softmax'])

 

graph_io.write_graph(constant_graph, '.', file_name, as_text=False)

 

Something I am doing wrong?

 

Thanx

 

S&|

 

p.s.: Just in case my model & the trained parameters are at

 

https://drive.google.com/open?id=1oJQIdpLn-wCjhARR6qxYlWwdyhHYcjy8
0 Kudos
idata
Employee
495 Views

@sseres, looks like the model was built with Keras and then you use tenssorflow to convert all variable to constants. Can you check if Keras is creating a nn.dropout or layers.dropout layer for you? I am not sure if an how .convert_variables_to_constants deals with tensorflow.nn.dropout, but I do know that freeze_graph does not like this op/layer. i.e. it skips the dropout layer and leaves behind the placeholder (& it's variable).

 

Here's an example project that attempts to deal with this issue. It's a work in progress, so I can't prove this solution works perfectly (especially since NCSDK doesn't yet support placeholder_with_defaults).

 

https://github.com/ashwinvijayakumar/ncappzoo/commit/0ff421e2f7363a4b369f0bbf95fedfc4b1164720#diff-414bbfe1e572ab483afc57fb07263431

 

Some tips for debugging your model

 

Use https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/import_pb_to_tensorboard.py to visualize your pb file, and check if you still see a placeholder with variable in the graph.

 

Update: freeze_graph uses .convert_variables_to_constants for converting variable to constants, but it also does other house cleaning tasks like removing device specific nodes (training related layers/ops).

0 Kudos
Reply