Age and Gender from Facenet

[idata]

Hello folks!

I'm an EE who in his free time tinkers with Python and in the recent time trying to learn about neural networks and NCS. At the time, I'm working on a project that tries to guess person's age and gender and this is what I got so far:

I'm using this project project as a backbone. I'm using author's pretrained network as it does exactly what I need. My end goal is to make it run on NCS. How can I do it?

I can see that project is based on David Sandberg's Facenet and there is great example on ncappzoo that runs Facenet on NCS flawlessly. I thought it will be easy to convert the network I'm using the same way the example explains. I manage to compile my graph and load it to the NCS, just like ncappzoo does. But I can't achieve what the original project does - to read out age and gender estimation. The output, userobj = graph.GetResult() returns an np array that, I assume, is 128 points of facial landmarks. The original project does something else:

gender = tf.argmax(tf.nn.softmax(gender_logits), 1) age_ = tf.cast(tf.constant([i for i in range(0, 101)]), tf.float32) age = tf.reduce_sum(tf.multiply(tf.nn.softmax(age_logits), age_), axis=1)this is something I can't read out NCS graph. Being self-thought, I don't have enough experience to dig deeper, so any help would be appreciated.

Cheers,

[idata]

@wesperso It looks like the original project uses dlib face detection and then passes the detected face to a Tensorflow gender and age network. I haven't used that specific TensorFlow age and gender models, but it looks like the gender model is testing how "male" a person is. Anything under 0.50 confidence score is a female and everything else is considered male https://github.com/yu4u/age-gender-estimation/blob/master/demo.py#L110. This is similar to the Caffe gender model that is supported by the NCSDK.

The Age network that is used in the project seems to return the highest confidence age score for classes (0-100) for each detected face.

This is just by looking at the code. I haven't tried this porting this particular project to the NCS yet. There are existing age and gender Caffe networks that currently work on the NCS. There is an ncappzoo app that uses OpenCV lbp face detect and the gender and age networks to predict gender and age based on the detected face. If you are using NCSDK 2, you can find the project here and if you are using NCSDK 1 you can use this link.

[idata]

@Tome_at_Intel thanks for the answer! You got it right, the program uses dlib 68-point facial recognition but before passing it to TF network it runs it through the face aligner made by Adrian Rosenbrock from PyImageSearch.

The project you linked above does the similar thing but using Keras, what makes it un-deployable at NCS at the time. Thanks for the links to Caffe networks on ncappzoo, I´m definitely going to give them a try. Though I'm not that confident when it comes to C++ … anyway, I'll write here how it goes.

BTW, how would you port this project to NCS? From what I understand, my model is made up of three files: .index, .meta and .data, and using them and program mvNCCompile I am able to generate something called _graph_ that will run on NCS. Why doesn't this graph return the same age and gender outputs as the original program does?

[idata]

@wesperso Why doesn't this graph return the same age and gender outputs as the original program does?

You mentioned that the line of code output, userobj = graph.GetResult() returns an array of 128 values? Does the original model only return two results, one for age and one for gender? I'm curious what the exact size of an array you get back from your model (NCS/NCSDK and on Tensorflow).

[idata]

not that I can tell. It returns various kinds of tensors, which I'm not experienced enough to dissect. Here's the crucial code:

sess = tf.Session() images_pl = tf.placeholder(tf.float32, shape=[None, 160, 160, 3], name='input_image') images_norm = tf.map_fn(lambda frame: tf.image.per_image_standardization(frame), images_pl) train_mode = tf.placeholder(tf.bool) age_logits, gender_logits, _ = inception_resnet_v1.inference(images_norm, keep_probability=0.8, phase_train=train_mode, weight_decay=1e-5) gender = tf.argmax(tf.nn.softmax(gender_logits), 1) age_ = tf.cast(tf.constant([i for i in range(0, 101)]), tf.float32) age = tf.reduce_sum(tf.multiply(tf.nn.softmax(age_logits), age_), axis=1) init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer()) sess.run(init_op) saver = tf.train.Saver() ckpt = tf.train.get_checkpoint_state("models/best_models/") saver.restore(sess, ckpt.model_checkpoint_path) ages,genders = sess.run([age, gender], feed_dict={images_pl: faces, train_mode: False})

The last line returns arrays of age and gender for each detected person on the picture. I just can't see similarity between this and what example with NCS does.

On the other side, I checked out gender_age_lpd example as you suggested. It works fine but it seams that I need two NCS to run it completely. Can this be somehow mitigated? Do you think that network I'm using above will eventually need two sticks if I ever make it run on NCS?

Regards,

W

[idata]

@wesperso If you use the NCSDK2 version of the gender_age_lbp code, (meaning you would have to install ncsdk version 2) you can run it on one NCS device (or two if you want). NCSDK 2 has a feature that enabled allocating multiple graphs on a single device. However there is an issue with virtual machines and NCSDK 2 at the moment, so if you are using a VM, I wouldn't recommend it with the current NCSDK (2.05). Dedicated machines should be fine.

[idata]

Hello, any news if the issue has been resolved? (issue on ncsdk on vm)

[idata]

@wesperso This issue is still being worked on. Thank you for your patience!

[idata]

@wesperso I tried to compile this model for NCS. To do this, I added a named output node to the model's graph. Then I freeze the model for inference. However, this model won't be able to run on the NCSDK, since the TF graph contains operations that haven't support by NCSDK. (details here)

[idata]

Hi, just want to mention that with the open vino toolkit, the model https://github.com/yu4u/age-gender-estimation/ can be converted to IR and used with the NCS. One thing to notice is to add -b 1 when running the model optimizer command