Mobilenet SSD conversion error from Keras to Tensorflow to Movidius

[idata]

Hello, I have a custom trained keras mobilenet model which i have converted to a tensorflow model successfully. I am now trying to convert it to movidius but I am running into this problem:

robert@atreus:~$ mvNCCheck -in input_1 -on predictions/concat dev/converter/TF_Model/tf_model.meta /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(False, "Layer type not supported by Convolution: " + obj.type) mvNCCheck v02.00, Copyright @ Intel Corporation 2017 ****** Info: No Weights provided. inferred path: dev/converter/TF_Model/tf_model.data-00000-of-00001****** dev/converter/TF_Model/tf_model.meta 2019-02-27 11:27:46.187790: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-02-27 11:27:46.209947: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2019-02-27 11:27:46.210526: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x4a0f550 executing computations on platform Host. Devices: 2019-02-27 11:27:46.210541: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined> WARNING:tensorflow:From /home/robert/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. Traceback (most recent call last): File "/usr/local/bin/mvNCCheck", line 239, in <module> quit_code = check_net(args.network, args.image, args.inputnode, args.outputnode, args.nshaves, args.inputsize, args.weights, args) File "/usr/local/bin/mvNCCheck", line 206, in check_net load_ret = load_network(args, parser, myriad_config) File "/usr/local/bin/ncsdk/Controllers/Scheduler.py", line 72, in load_network parsedLayers = p.parse(arguments) File "/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlow.py", line 673, in parse parsed_op = opParser(op, operations, dummy_feed_dict) File "/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/FusedBatchNorm.py", line 36, in load scale = np.reciprocal(np.sqrt(variance)) * scale_param ValueError: operands could not be broadcast together with shapes (0,) (32,)

This is the original keras model that I am using to train:

https://github.com/tanakataiki/ssd_kerasV2/blob/master/model/ssd300MobileNet.py

After training I am converting to a tensorflow graph like this:

input_shape = (300, 300, 3) mobilenet_model = SSD(input_shape, num_classes=1) mobilenet_model.load_weights('mobilenet_test/weights.32-3.41.hdf5', by_name=True) def keras_to_tf( ): tf_model_path = './TF_Model/tf_model' saver = tf.train.Saver() with K.get_session() as sess: K.set_learning_phase(0) saver.save(sess, tf_model_path) return True keras_to_tf( )

[idata]

Apparently there's a config file that needs to be used:

https://ncsforum.movidius.com/discussion/1588/ncsdk-2-10-option-tf-ssd-config#latest

after doing so, I get input/output node name errors even after converting from .meta to .pb (freezing the graph)

@Jesus_at_Intel could it be that my model structure is wrong?

https://github.com/tanakataiki/ssd_kerasV2/blob/master/model/ssd300MobileNet.py

[idata]

I am stuck on this issue still, I am trying multiple ways of converting mobilenet to movidius and with no luck

Here are some examples.

robert@atreus:~/dev/converter$ mvNCCompile mobilenet_test/groceries/model.ckpt.meta --tf-ssd-config tf_ssd.conf /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(False, "Layer type not supported by Convolution: " + obj.type) mvNCCompile v02.00, Copyright @ Intel Corporation 2017 ****** Info: No Weights provided. inferred path: mobilenet_test/groceries/model.ckpt.data-00000-of-00001****** mobilenet_test/groceries/model.ckpt.meta 2019-02-27 16:18:54.843088: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-02-27 16:18:54.865891: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2019-02-27 16:18:54.866276: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x518eb40 executing computations on platform Host. Devices: 2019-02-27 16:18:54.866307: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined> WARNING:tensorflow:From /home/robert/.local/lib/python3.5/site-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version. Instructions for updating: Use standard file APIs to check for files with this prefix. [Error 5] Toolkit Error: Stage Details Not Supported: FeatureExtractor/MobilenetV1/MobilenetV1/Conv2d_0/BatchNorm/batchnorm/mul_1 robert@atreus:~/dev/converter$ mvNCCompile mobilenet_test/output_graph.pb -in DecodeJpeg -on final_result /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(False, "Layer type not supported by Convolution: " + obj.type) mvNCCompile v02.00, Copyright @ Intel Corporation 2017 mobilenet_test/output_graph.pb 2019-02-27 16:37:54.877150: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-02-27 16:37:54.897864: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2019-02-27 16:37:54.898292: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x631fef0 executing computations on platform Host. Devices: 2019-02-27 16:37:54.898307: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined> 2019-02-27 16:37:55.078046: W tensorflow/core/framework/op_def_util.cc:355] Op BatchNormWithGlobalNormalization is deprecated. It will cease to work in GraphDef version 9. Use tf.nn.batch_normalization(). [Error 5] Toolkit Error: Stage Details Not Supported: Dynamic input shape not supported.

These graphs were trained with the Object Detection methods Tensorflow provides on their Models/Research libraries

[idata]

input_1 conv1_pad conv1 conv1_bn conv1_relu conv_pad_1 conv_dw_1 conv_dw_1_bn conv_dw_1_relu conv_pw_1 conv_pw_1_bn conv_pw_1_relu conv_pad_2 conv_dw_2 conv_dw_2_bn conv_dw_2_relu conv_pw_2 conv_pw_2_bn conv_pw_2_relu conv_pad_3 conv_dw_3 conv_dw_3_bn conv_dw_3_relu conv_pw_3 conv_pw_3_bn conv_pw_3_relu conv_pad_4 conv_dw_4 conv_dw_4_bn conv_dw_4_relu conv_pw_4 conv_pw_4_bn conv_pw_4_relu conv_pad_5 conv_dw_5 conv_dw_5_bn conv_dw_5_relu conv_pw_5 conv_pw_5_bn conv_pw_5_relu conv_pad_6 conv_dw_6 conv_dw_6_bn conv_dw_6_relu conv_pw_6 conv_pw_6_bn conv_pw_6_relu conv_pad_7 conv_dw_7 conv_dw_7_bn conv_dw_7_relu conv_pw_7 conv_pw_7_bn conv_pw_7_relu conv_pad_8 conv_dw_8 conv_dw_8_bn conv_dw_8_relu conv_pw_8 conv_pw_8_bn conv_pw_8_relu conv_pad_9 conv_dw_9 conv_dw_9_bn conv_dw_9_relu conv_pw_9 conv_pw_9_bn conv_pw_9_relu conv_pad_10 conv_dw_10 conv_dw_10_bn conv_dw_10_relu conv_pw_10 conv_pw_10_bn conv_pw_10_relu conv_pad_11 conv_dw_11 conv_dw_11_bn conv_dw_11_relu conv__11 conv_11_bn conv_11_relu conv_pad_12 conv_dw_12 conv_dw_12_bn conv_dw_12_relu conv_pw_12 conv_pw_12_bn conv_pw_12_relu conv_pad_13 conv_dw_13 conv_dw_13_bn conv_dw_13_relu conv__13 conv_13_bn conv_13_relu conv_pad_14_1 conv__14_1 conv_14_bn_1 conv_14_relu_1 conv__14_2 conv_14_bn_2 conv_14_relu_2 conv_pad_15_1 conv__15_1 conv_15_bn_1 conv_15_relu_1 conv__15_2 conv_15_bn_2 conv_15_relu_2 conv_pad_16_1 conv__16_1 conv_16_bn_1 conv_16_relu_1 conv__16_2 conv_16_bn_2 conv_16_relu_2 conv_pad_17_1 conv__17_1 conv_17_bn_1 conv_17_relu_1 conv__17_2 conv11_mbox_conf conv13_mbox_conf conv14_2_mbox_conf conv15_2_mbox_conf conv16_2_mbox_conf conv17_2_mbox_conf conv11_mbox_loc conv13_mbox_loc conv14_2_mbox_loc conv15_2_mbox_loc conv16_2_mbox_loc conv17_2_mbox_loc conv11_mbox_conf_flat conv13_mbox_conf_flat conv14_2_mbox_conf_flat conv15_2_mbox_conf_flat conv16_2_mbox_conf_flat conv17_2_mbox_conf_flat conv11_mbox_loc_flat conv13_mbox_loc_flat conv14_2_mbox_loc_flat conv15_2_mbox_loc_flat conv16_2_mbox_loc_flat conv17_2_mbox_loc_flat mbox_conf mbox_loc mbox_conf_logits conv11_mbox_priorbox conv13_mbox_priorbox conv14_2_mbox_priorbox conv15_2_mbox_priorbox conv16_2_mbox_priorbox conv17_2_mbox_priorbox mbox_loc_final mbox_conf_final mbox_priorbox predictions

This is my model structure, I'm not sure if there's any layers that are not compatible with mvNCCompile and if so, how to fix it.

[idata]

Hi @robert_osmosis

Thanks so much for your patience.

There are sometimes issues when converting custom models trained in Keras to Tensorflow to NCS Graph. And the "stage details details not supported:" errors means that there are layers or operations that you're using that are still unsupported by the NCSDK. By just looking at these error messages, that's all I'm able to tell. Can you share your model and procedures with us? I want to try to convert your model on my side and I'll share the results with you.

Also, try to enable the debug option in the Tensorflow parser, this could provide us additional information about the error. You will need to set the debug flag to True in the following file as sudo:

/usr/local/bin/ncsdk/Controllers/Parsers/TensorFlow.py

Sincerely,

Sahira

[idata]

Thank you for the reply @Sahira_at_Intel

https://drive.google.com/drive/folders/1wXGvPXLqf5iPrHk-gozKDyu9CWrPErHH?usp=sharing

Here is the model, which I converted doing this:

with K.get_session() as sess: K.set_learning_phase(0) print("wat") saver.save(sess, tf_model_path) frozen = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ["predictions/concat"]) graph_io.write_graph(frozen, './', 'inference_graph.pb', as_text=False)

After setting the debug flag to True, this is the output

robert@atreus:~/dev/converter$ mvNCCompile inference_graph.pb --tf-ssd-config tf_ssd.conf /usr/local/bin/ncsdk/Controllers/Parsers/TensorFlowParser/Convolution.py:47: SyntaxWarning: assertion is always true, perhaps remove parentheses? assert(False, "Layer type not supported by Convolution: " + obj.type) mvNCCompile v02.00, Copyright @ Intel Corporation 2017 inference_graph.pb 2019-03-07 13:44:57.777974: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2019-03-07 13:44:57.799708: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2019-03-07 13:44:57.800257: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x6049030 executing computations on platform Host. Devices: 2019-03-07 13:44:57.800284: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): <undefined>, <undefined> [Error 13] Toolkit Error: Provided OutputNode/InputNode name does not exist or does not match with one contained in model file Provided: . Could not infer output node names. Default names:num_detections,detection_scores,detection_classes,detection_boxes

[idata]

Hi @robert_osmosis

Since NCSDK 2.10 is compatible with the SSD_mobilenet_v1/v2_coco models, you can try re-training a model based on SSD_mobilenet_v1/v2_coco with your data. Afterwards, you can try converting that newly retrained model to an NCS graph. The SSD_mobilenet_v1/v2_coco models can be found here . Please let me know if you were able to successfully convert the model + deploy.

Best Regards,

Sahira

[idata]

Model compiles but does not detect very well at all. Has the Intel team had a chance to test TensorFlow MobileNetSSD and had better result?