Community
cancel
Showing results for 
Search instead for 
Did you mean: 
idata
Community Manager
526 Views

net.forward() very slow on Raspberry Pi with sequential keras model

Hello Movidius NN Community -

 

I built a very simple keras model (see below). Trained it on CIFAR10. Froze my model and converted it to .bin and .xml with:

 

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model keras_CIFAR10_frozen.pb --input_shape [1,32,32,3] --data_type FP16 --log_level WARNING

 

When I use this model on my Mac with the CPU as the prefered target, the net.forward() takes 0.1s.

 

When I run it on the rasp with the Myriad as target, the net.forward() takes 2.2s… the good thing is that it is working ;-) the bad news is that it's very slow.

 

Am I missing something?

 

The object detection example works at 7 fps, is there any parameters I am missing?

 

I am freezing my graph as follow:

 

python /Users/nicolasmaquaire/Dropbox/KLoop/Dev/190209_MLEnvs/env/lib/python3.7/site-packages/tensorflow/python/tools/freeze_graph.py \

 

--input_meta_graph=keras_CIFAR_Seq_Sparse.ckpt.meta \

 

--input_checkpoint=keras_CIFAR_Seq_Sparse.ckpt \

 

--output_graph=keras_CIFAR_Seq_Sparse_frozen.pb \

 

--output_node_names="activation_1/Softmax" \

 

--input_binary=true

 

and converting it with:

 

python3 /opt/intel/openvino/deployment_tools/model_optimizer/mo_tf.py --input_model keras_CIFAR_Seq_Sparse_frozen.pb --input_shape [1,32,32,3] --log_level WARNING --data_type FP16

 

Any clues are really welcome.

 

Thank you!

 

My model:

 

model = tf.keras.models.Sequential() model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(64, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(128, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.BatchNormalization(input_shape=(target_size, target_size, 3))) model.add(tf.keras.layers.Conv2D(256, (5, 5), padding='same', activation='relu')) model.add(tf.keras.layers.MaxPooling2D(pool_size=(2, 2), strides=(2,2))) model.add(tf.keras.layers.Dropout(0.25)) model.add(tf.keras.layers.Flatten()) model.add(tf.keras.layers.Dense(256)) model.add(tf.keras.layers.Activation('relu')) model.add(tf.keras.layers.Dropout(0.5)) model.add(tf.keras.layers.Dense(10)) model.add(tf.keras.layers.Activation('softmax'))
0 Kudos
2 Replies
idata
Community Manager
150 Views

Hi @nicmaq

 

Thanks for providing your model, I'll try to reproduce and get back to you with results.

 

Best Regards,

 

Sahira
idata
Community Manager
150 Views

Thank you Sahira - I decided to go with another option, as post-training quantization is not compatible with the edge-tpu and I am trying to compare both option. I create a new thread. Thank you agai for your help. Much appreciated.

Reply