net.forward() very slow on Raspberry Pi with sequential keras model

[idata]

Hello Movidius NN Community -

I built a very simple keras model (see below). Trained it on CIFAR10. Froze my model and converted it to .bin and .xml with:

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model keras_CIFAR10_frozen.pb --input_shape [1,32,32,3] --data_type FP16 --log_level WARNING

When I use this model on my Mac with the CPU as the prefered target, the net.forward() takes 0.1s.

When I run it on the rasp with the Myriad as target, the net.forward() takes 2.2s… the good thing is that it is working ;-) the bad news is that it's very slow.

Am I missing something?

The object detection example works at 7 fps, is there any parameters I am missing?

I am freezing my graph as follow:

python /Users/nicolasmaquaire/Dropbox/KLoop/Dev/190209_MLEnvs/env/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py \

--input_meta_graph=keras_CIFAR_Seq_Sparse.ckpt.meta \--input_checkpoint=keras_CIFAR_Seq_Sparse.ckpt \--output_graph=keras_CIFAR_Seq_Sparse_frozen.pb \--output_node_names="activation_1/Softmax" \--input_binary=true

and converting it with:

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model keras_CIFAR_Seq_Sparse_frozen.pb --input_shape [1,32,32,3] --log_level WARNING --data_type FP16

Any clues are really welcome.

Thank you!

My model:

model = tf.keras.models.Sequential() model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(64, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(128, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(256, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.Flatten()) model.add(tf.keras.layers.Dense(256)) model.add(tf.keras.layers.Activation('relu')) model.add(tf.keras.layers.Dropout(0.5)) model.add(tf.keras.layers.Dense(10)) model.add(tf.keras.layers.Activation('softmax'))

[idata]

Hi @nicmaq

Thanks for providing your model, I'll try to reproduce and get back to you with results.

Best Regards,

Sahira

[idata]

Thank you Sahira - I decided to go with another option, as post-training quantization is not compatible with the edge-tpu and I am trying to compare both option. I create a new thread. Thank you agai for your help. Much appreciated.