Re: ENet Performance Issues

idata · ‎05-02-2018

I am trying to run the ENet (with Tensorflow) on the NCS. Due to unsupported operations I had to modify the original architecture but finally I was able to compile the network. In the following I will state my procedure to receive a NCS graph file:

Training.
Restore model, remove layers that are not needed for inference and save again.
I am using the freeze_graph tool to merge the inference graph and weights to a .pb file.
Now I am using the graph_transform tool to optimize the frozen model (note: I am also using the flatten_atrous_conv transform otherwise I receive the error that "SpaceToBatchND" is not supported)
The transformed graph is then compiled via mvNCCompile

As I am interested in the performance of the NCS I ran mvNCProfile and noticed that two "simple" operations (mul and add) take quite a long time and especially the softmax operation in the end. I appreciate any guess or solution to this performance loss.

Here is a part of the profile output:

...
188 ENet/Relu_127 0.0 32.7 0.169
189 ENet/mul_127 0.0 126.8 0.044
190 ENet/add_63 0.0 109.9 0.050
191 ENet/Relu_129/ENet/mul_129 0.0 49.6 0.113
192 ENet/add_64 0.0 29.8 0.046
193 ENet/Relu_131/ENet/mul_131 1.7 11.4 35.854
194 ENet/add_65 0.0 26.5 0.052
195 ENet/bottleneck2_8_batch_norm3/batchnorm/add_1 0.0 8.4 0.182
196 ENet/Relu_133 0.0 122.2 0.045
197 ENet/Relu_132 0.0 131.0 0.042
198 ENet/add_66 0.0 108.9 0.051
...
310 ENet/add_104 0.0 29.6 0.047
311 ENet/bottleneck3_8_batch_norm2/batchnorm/add_1 1.7 11.3 35.955
312 ENet/Relu_211 0.0 31.0 0.045
313 ENet/Relu_210 0.0 33.9 0.041
314 ENet/add_105 0.0 29.0 0.048
315 ENet/bottleneck3_8_batch_norm3/batchnorm/add_1 0.0 5.0 0.304
316 ENet/Relu_213 0.0 113.9 0.049
317 ENet/Relu_212 0.0 131.9 0.042
318 ENet/add_106 0.0 105.8 0.052
319 ENet/Relu_215 0.0 131.4 0.042
320 ENet/Relu_214 0.0 130.2 0.043
...
353 ENet/bottleneck5_0_last_prelu 0.0 123.8 0.710
354 ENet/bottleneck5_1_prelu1 0.4 229.5 0.383
355 ENet/bottleneck5_1_prelu2 0.8 276.1 0.717
356 ENet/bottleneck5_1_prelu4 0.4 36.8 0.601
357 ENet/bottleneck5_1_last_prelu 0.0 122.9 0.716
358 ENet/fullconv/BiasAdd 0.0 13.0 6.868
359 ENet/Softmax 0.4 0.6 451.514

idata · ‎05-03-2018

@toko It could be due to a variety of issues. Can you provide your model so I can take a look at it?

idata · ‎05-08-2018

@Tome_at_Intel thank you for your reply. Here (https://files.fm/u/9hhmpftq) you can find the model (.pb) file. The input and output layers are: -in=input -on=probabilities

idata · ‎05-10-2018

@toko I am unable to run the model. I get a shape error with NCSDK 2.4 and placeholder issue with NCSDK 1.12. Can you let me know which NCSDK version you were using? Also can you double check that this is the model you used? The command I tried was mvNCCompile model.pb -s 12 -in input -on probabilities. Thanks!

NCSDK 2.4 error:

raise ValueError("Shapes %s and %s are not compatible" % (self, other)) ValueError: Shapes (12, 15, 32) and (12, 15, 16, 0) are not compatible

NCSDK 1.12 error:


InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float and shape [1,90,120,3]
     [[Node: input = Placeholder[dtype=DT_FLOAT, shape=[1,90,120,3], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

idata · ‎05-15-2018

@Tome_at_Intel I am using the same command (with the uploaded file) and it is giving me a graph file and a warning (You are using a large type. Consider reducing your data sizes for best performance).

My specs are: Ubuntu 16.04 LTS, Python 2.7, NCSDK 1.12, Tensorflow 1.4