Concat Layer Support for TensorFlow Models

[idata]

I am using some of the SSD Models from the TensorFlow Detection Model Zoo (SSD-MobileNetv1 and SSD-Inceptionv2). I need to get them working on the NCS for a fast prototype.

I know the models are not working with the stick with the available trained files (.pb) from TensorFlow Zoo due to the Anchor Box generator and the concat layer.I tried to isolate the Anchor Box generator (to be done out of the stick with the box decoding step) in order to verify what the NCS can do in compiling the model itself.I made some changes to how the output is obtained:

1. Remove the box decoding step from the model in order to suppress the Anchor box generation which is not working on the stick (maybe done outside the stick).
2. Add a CONCAT Layer to combine the box_predictions and the class_predictions in order to get a single output to work with the SDK.
3. Add a dummy layer at the end (MAX_POOL with stride 1 and kernel 1) to workaround the issue of last layer being CONCAT.
4. Prepare the inputs to each CONCAT layer to be in 4D.
5. Using the SDK 2.08 and enabled the TensorFlow new_parser. I verified the new_parser is working by running the mvNCCheck for the inceptionv3 as provided in the examples of the SDK.

The box_predictions output node is named "bboxes" and the class_predictions output node is named "scores".

I tried running the following command mvNCCheck ssd_mobilenet_v1_ncs.pb -s 12 and the resulting log is attached.I also tried to run with both -on scores and -on bboxes but I got all FAIL.The GOOD is the model compiles successfully, but the mvNCCheck report Failures.

@Tome_at_Intel , Could you please give me any help about this issue? Why CONCAT is working fine for the ssd_mobilenet in CAFFE and not working in TensorFlow?

I attached both the frozen model pb graph and a log file captured from running the mvNCCheck (I added some debug logs for the concat and dumping the layer names).I appreciate your feedback.

Regards,

Ahmed

[idata]

I ran the command mvNCCheck ssd_mobilenet_v1_ncs.pb -s 12 -on BoxPredictor_0/Reshape and I got the FAIL for all results. So, what is wrong with the reshape in this case?

The reshape is supposed to reshape the array from [1, 19, 19, 12] to [1, 1, 1083, 4]. I think the error in the reshape layer most probably the cause of the problem.@Tome_at_Intel Any thoughts?

[idata]

@ahmed.ezzat There currently isn't support for SSD Mobilenet at the moment on the current version of the NCSDK (2.08.01). NCSDK doesn't have all the needed layers/ops implemented for SSD MobileNet for TensorFlow so that's probably the reason it is not working for you.

[idata]

@Tome_at_Intel I know that it is not supported (I mentioned that in my first comment). I am trying to help myself by workaround the issue.

After re-generating the '.pb' graph file (removing the bbox decoding and the Anchor Box generation) I get it compiled successfully!The modified graph (attached) contains only operators supported by the NCSDK.The problem starts at the output of a reshape layer, which is strange, as I understand it is supported for both Caffe and TensorFlow in the new parser.To be specific, Is there a problem or limitations in the NCSDK corresponding to TensorFlow Reshape Layer?

[idata]

We are very interested in this.

@Tome_at_Intel can you respond as this would really help us.

[idata]

@ahmed.ezzat @djaenicke Looks like the issue may be in the concat operation. At the moment, the NCSDK doesn't have support for axis specific concat yet. This is probably why the shapes are wrong after the concat.

> CONCAT : concat:0 --> (1, 4, 1, 1917) ======================================== Inputs 0 : ( BoxPredictor_0/Reshape:0 , (1, 4, 1, 1083) ) 1 : ( BoxPredictor_1/Reshape:0 , (1, 4, 1, 600) ) 2 : ( BoxPredictor_2/Reshape:0 , (1, 4, 1, 150) ) 3 : ( BoxPredictor_3/Reshape:0 , (1, 4, 1, 54) ) 4 : ( BoxPredictor_4/Reshape:0 , (1, 4, 1, 24) ) 5 : ( BoxPredictor_5/Reshape:0 , (1, 4, 1, 6) ) Concat Shape --> (1, 4, 1, 1917) Concat Axis --> 3

[idata]

Hi @Tome_at_Intel

thank you - is there any roadmap to add support for this?

[idata]

@djaenicke I apologize, but at the moment I don't have a roadmap for this feature that I can share.

[idata]

This is a bit of a bummer! In marketing materials you claim that tensor flow models are supported and you cannot even provide a workaround. How come?