Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Community Manager
230 Views

ENet Performance Issues

I am trying to run the ENet (with Tensorflow) on the NCS. Due to unsupported operations I had to modify the original architecture but finally I was able to compile the network. In the following I will state my procedure to receive a NCS graph file:

 

  1. Training.
  2. Restore model, remove layers that are not needed for inference and save again.
  3. I am using the freeze_graph tool to merge the inference graph and weights to a .pb file.
  4. Now I am using the graph_transform tool to optimize the frozen model (note: I am also using the flatten_atrous_conv transform otherwise I receive the error that "SpaceToBatchND" is not supported)
  5. The transformed graph is then compiled via mvNCCompile

 

As I am interested in the performance of the NCS I ran mvNCProfile and noticed that two "simple" operations (mul and add) take quite a long time and especially the softmax operation in the end. I appreciate any guess or solution to this performance loss.

 

Here is a part of the profile output:

... 188 ENet/Relu_127 0.0 32.7 0.169 189 ENet/mul_127 0.0 126.8 0.044 190 ENet/add_63 0.0 109.9 0.050 191 ENet/Relu_129/ENet/mul_129 0.0 49.6 0.113 192 ENet/add_64 0.0 29.8 0.046 193 ENet/Relu_131/ENet/mul_131 1.7 11.4 35.854 194 ENet/add_65 0.0 26.5 0.052 195 ENet/bottleneck2_8_batch_norm3/batchnorm/add_1 0.0 8.4 0.182 196 ENet/Relu_133 0.0 122.2 0.045 197 ENet/Relu_132 0.0 131.0 0.042 198 ENet/add_66 0.0 108.9 0.051 ... 310 ENet/add_104 0.0 29.6 0.047 311 ENet/bottleneck3_8_batch_norm2/batchnorm/add_1 1.7 11.3 35.955 312 ENet/Relu_211 0.0 31.0 0.045 313 ENet/Relu_210 0.0 33.9 0.041 314 ENet/add_105 0.0 29.0 0.048 315 ENet/bottleneck3_8_batch_norm3/batchnorm/add_1 0.0 5.0 0.304 316 ENet/Relu_213 0.0 113.9 0.049 317 ENet/Relu_212 0.0 131.9 0.042 318 ENet/add_106 0.0 105.8 0.052 319 ENet/Relu_215 0.0 131.4 0.042 320 ENet/Relu_214 0.0 130.2 0.043 ... 353 ENet/bottleneck5_0_last_prelu 0.0 123.8 0.710 354 ENet/bottleneck5_1_prelu1 0.4 229.5 0.383 355 ENet/bottleneck5_1_prelu2 0.8 276.1 0.717 356 ENet/bottleneck5_1_prelu4 0.4 36.8 0.601 357 ENet/bottleneck5_1_last_prelu 0.0 122.9 0.716 358 ENet/fullconv/BiasAdd 0.0 13.0 6.868 359 ENet/Softmax 0.4 0.6 451.514

 

0 Kudos
4 Replies
Highlighted
Community Manager
2 Views

Re: ENet Performance Issues

@toko It could be due to a variety of issues. Can you provide your model so I can take a look at it?

0 Kudos
Highlighted
Community Manager
2 Views

Re: ENet Performance Issues

@Tome_at_Intel thank you for your reply. Here (https://files.fm/u/9hhmpftq) you can find the model (.pb) file. The input and output layers are: -in=input -on=probabilities

0 Kudos
Highlighted
Community Manager
2 Views

Re: ENet Performance Issues

@toko I am unable to run the model. I get a shape error with NCSDK 2.4 and placeholder issue with NCSDK 1.12. Can you let me know which NCSDK version you were using? Also can you double check that this is the model you used? The command I tried was mvNCCompile model.pb -s 12 -in input -on probabilities. Thanks!

 

NCSDK 2.4 error:

 

raise ValueError("Shapes %s and %s are not compatible" % (self, other)) ValueError: Shapes (12, 15, 32) and (12, 15, 16, 0) are not compatible

 

NCSDK 1.12 error:

 

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float and shape [1,90,120,3] [[Node: input = Placeholder[dtype=DT_FLOAT, shape=[1,90,120,3], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
0 Kudos
Highlighted
Community Manager
2 Views

Re: ENet Performance Issues

@Tome_at_Intel I am using the same command (with the uploaded file) and it is giving me a graph file and a warning (You are using a large type. Consider reducing your data sizes for best performance).

 

My specs are: Ubuntu 16.04 LTS, Python 2.7, NCSDK 1.12, Tensorflow 1.4

0 Kudos