Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Community Manager
341 Views

Compiling quantized net

Hey, i'm a student doing a project with the ncs trying to compare it to other solutions. Since it's architecture is vector based and it supports int8 operations it would get a great edge if i were to quantize the network.

 

I am only using pre-trained models and i was trying to use the auto-quantization through :https://www.tensorflow.org/versions/r1.3/performance/quantization.

 

Tried it with inception v3 and inception_resnet_v2, but i cant seem to compile the quantized outputs.

 

tried to plain copy like thus:

 

bazel-bin/tensorflow/tools/graph_transforms/transform_graph \

 

--in_graph=inception_v3_frozen.pb \

 

--outputs="InceptionV3/Predictions/Reshape_1" --out_graph=/tmp/quantized_graph.pb \

 

--transforms='add_default_attributes strip_unused_nodes(type=float, shape="1,299,299,3")

 

remove_nodes(op=Identity, op=CheckNumerics) fold_constants(ignore_errors=true)

 

fold_batch_norms fold_old_batch_norms quantize_weights quantize_nodes

 

strip_unused_nodes sort_by_execution_order'

 

threw :

 

[Error 5] Toolkit Error: Stage Details Not Supported: Max

 

so i tried to limit the optimizations as i thought some might not have been implemented:

 

bazel-bin/tensorflow/tools/graph_transforms/transform_graph \

 

--in_graph=inception_v3_frozen.pb \

 

--outputs="InceptionV3/Predictions/Reshape_1" --out_graph=quantized_graph2.pb \

 

--transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") quantize_weights quantize_nodes'

 

threw:

 

Traceback (most recent call last):

 

File "/usr/local/bin/mvNCCompile", line 156, in

 

create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights, args.explicit_concat, args.ma2480, args.scheduler, args)

 

File "/usr/local/bin/mvNCCompile", line 137, in create_graph

 

load_ret = load_network(args, parser, myriad_config)

 

File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 82, in load_network

 

parse_ret = parse_tensor(arguments, myriad_conf)

 

File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 315, in parse_tensor

 

item_shape = output_item.shape.as_list()

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/tensor_shape.py", line 820, in as_list

 

raise ValueError("as_list() is not defined on an unknown TensorShape.")

 

ValueError: as_list() is not defined on an unknown TensorShape.

 

then i made an account to make this post but before i did i tried with another graph:

 

bazel-bin/tensorflow/tools/graph_transforms/transform_graph --in_graph=inception_resnet_v2_frozen.pb \

 

--inputs="input" \

 

--outputs="InceptionResnetV2/Logits/Predictions" --out_graph=../i_res_v2_qt.pb \

 

--transforms=' strip_unused_nodes(type=float, shape="1,299,299,3") quantize_weights quantize_nodes'

 

threw:

 

File "/usr/local/bin/mvNCCompile", line 156, in

 

create_graph(args.network, args.inputnode, args.outputnode, args.outfile, args.nshaves, args.inputsize, args.weights, args.explicit_concat, args.ma2480, args.scheduler, args)

 

File "/usr/local/bin/mvNCCompile", line 137, in create_graph

 

load_ret = load_network(args, parser, myriad_config)

 

File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 82, in load_network

 

parse_ret = parse_tensor(arguments, myriad_conf)

 

File "/usr/local/bin/ncsdk/Controllers/TensorFlowParser.py", line 225, in parse_tensor

 

inputTensor = graph.get_tensor_by_name(inputnode + ':0')

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3654, in get_tensor_by_name

 

return self.as_graph_element(name, allow_tensor=True, allow_operation=False)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3478, in as_graph_element

 

return self._as_graph_element_locked(obj, allow_tensor, allow_operation)

 

File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 3520, in _as_graph_element_locked

 

"graph." % (repr(name), repr(op_name)))

 

KeyError: "The name 'input:0' refers to a Tensor which does not exist. The operation, 'input', does not exist in the graph."

 

thsi

 

now this seems to about the imput node but since i used the same compilation as the makefile and tried to preserve the input by the "--inputs="input" " option i dont know how to solve it.

 

While i'm at it i'd also like to try to use tinyolo if i can get this quantized and was wondering if anyone has any .pb models of it, tried to use the darkflow conversion but couldnt compile the output into a .graph either :neutral:

 

Anyhow sorry for the long post, let me know if any of you have encountered a similar issue.

 

Best regards, Nuno Magalhães.

Tags (1)
0 Kudos
1 Reply
Highlighted
Community Manager
10 Views

I'm encountering the same error.

 

as_list() is not defined on an unknown TensorShape.

 

Did you manage to solve it? I'd rather not retrain my network but I can't find a way to define the input node Tensorshape in the freeze_graph or transform_graph tools

0 Kudos