Labels missing in object_detection_demo_ssd_async demo

[Lange__Benjamin]

Hello everyone,

this is my software environment:
Windows 10
CMake 3.1
Python 3.5
Visual Studio 2017
OpenCV 4.0
Intel C++ Compiler 2017 Redistributable package for Windows
Neural Compute Stick (Myriad 2)

I converted a model from the Tensorflow detection model zoo (https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md) namely "ssd_mobilenet_v1_0_75_depth_300x300_coco14_sync_2018_07_03".

The demo I ran was "object_detection_demo_ssd_async demo". The demo worked fine but the labels were missing. Reviewing the source code it seems to me that I have to provide a labels file with the "-m" option. Providing a labels file is mentioned somewhere in the the docs for the demo (https://software.intel.com/en-us/articles/OpenVINO-IE-Samples#object-detection-SSD-showcase) as a side note. But an explicit example is missing or was overlooked (?).

The labels file is a simply a file where each label has its own line. I named it with the ending ".labels" since according to the source code it is the ending the program is looking for.

The command I issue looks like this:

Intel\OpenVINO\inference_engine_samples_2017\intel64\Release\object_detection_demo_ssd_async.exe -i cam -m F:\ssd_mobilenet_v1_0-75_300x300_coco14_ir_fp16\frozen_inference_graph.xml F:\ssd_mobilenet_v1_0-75_300x300_coco14_ir_fp16\coco.labels -d MYRIAD

Does anyone have a clue what might be going wrong here?

Best regards,

B. Lange

[Lange__Benjamin · Accepted solution]

Ok found my error, I did not read the code properly.

The name of the labels file needs to be named after the input model e.g. "frozen_inference_graph.labels".

Now everything works fine, case closed.

[Altmaier__Richard]

Hi, I am new to Intel's OpenVINO distribution and tensorflow models.  I have a simple question, where is the labels file to be found, which corresponds to any given tensorflow model, such as the ones at https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

Thanks! 

[SC_Huang]

Altmaier, Richard wrote:

Hi, I am new to Intel's OpenVINO distribution and tensorflow models.  I have a simple question, where is the labels file to be found, which corresponds to any given tensorflow model, such as the ones at https://github.com/tensorflow/models/blob/master/research/object_detecti...

Thanks! 

I have same question, even i've found each object of the IDs like below, I still cannot figure out the data structure of the labels file.

https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/

If someone can supply the label file or simply describe the structure of the lables file will be great

[nikos1]

Hello Huang,

The .labels file should have the same name as .bin and .xml for the sample applications in OpenVino to read.

For example: yolo_v3.bin, yolo_v3.xml , yolo_v3.labels all in the same folder.

The .labels files is a simple text file with each class name in every line like for example

kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
. . . 

Cheers,

Nikos

[Severine_H_Intel]

Dear all, 

in the labels file, you should withdraw the whitespace, otherwise, it considers: wine glass as 2 labels. You can replace it with an underscore for example. 

Best, 

Severine

[Lange__Benjamin]

Not all labels are used!

My label file for e.g. ssd_mobilenet_v1_0.75_depth_coco is attached to this post.

You have to remove the "doc" ending. I added it to be able to upload the file.

Do not remove any of the empty lines in the document! Otherwise you will get wrong label IDs.
You can derive the IDs from this file:

https://github.com/tensorflow/models/blob/master/research/object_detection/data/mscoco_label_map.pbtxt

It tells you which IDs are actually used and which are not.

How do you know which label map is used?

This information is written in the config, which can be found here: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync.config

You have to look at the very bottom of the config where it says:

"label_map_path: "PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt"

It took me a while to figure this out, because I could not find a source explaining this.

[nikos1]

Hello Benjamin,

I am not sure if I understand your question correctly but please note as Severine mentioned above : 

in the labels file, you should withdraw the whitespace, otherwise, it considers: wine glass as 2 labels. You can replace it with an underscore for example. 

Your attached frozen_inference_graph.labels.doc is not a good OpenVino labels file.

For example 

stop sign

should become something like

stop_sign

unless of course you changed the sample source code.

Also I am not sure how the current code will deal with the empty lines. Better to replace with an actual label like "empty" or :"no_label"

cheers,

nikos

[Lange__Benjamin]

I did not experience any problems with whitespaces. The empy lines just worked fine as well.

My problem was mainly to find the labels file and finding out which IDs were actually used.

I attached the changed file according to Nikos suggestions.

I cannot find the option to delete the previously uploaded file though.

[tay__lenelle]

Hi,

I would like to understand that after converting my tensorflow model to .xml and .bin files, i have successfully ran the object detection. However, there were no labels appearing with the bounding box. I would like to seek help in getting the labels please thank you.

[Olivero__Alberto]

Lange, Benjamin wrote:

I did not experience any problems with whitespaces. The empy lines just worked fine as well.

My problem was mainly to find the labels file and finding out which IDs were actually used.

I attached the changed file according to Nikos suggestions.

I cannot find the option to delete the previously uploaded file though.

Hi Benjamin, I'm using "ssd_mobilenet_v1_coco_2018_01_28" and the inference command as for your suggestion is

./object_detection_demo_ssd_async -i cam -m ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.xml ssd_mobilenet_OLI.labels -d MYRIAD -t 0.6

but I still see label #1, laber 36# and no names. Am I wrongly applying your suggestion ?

[Lange__Benjamin]

Hi Olivero,

are you using the label file I uploaded in the post you cited?

The command you used will show you no labels and it will not throw an error if you actually did not provide a label file.

Would you upload your label file so that I can have a look?

Kind regards

Benjamin

[Olivero__Alberto]

Hi Benjamin,

• I used your label file (the one where you added underscore in the post Tue, 02/05/2019 - 01:01).
• I just changed the file name to use the same name of my model as for blog suggest.
• I put it in the same folder of SSD_mobilenet model: ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/

BR

Alberto

[Lange__Benjamin]

Have you tried the command providing the path to the labels file like this:

./object_detection_demo_ssd_async -i cam -m ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.xml ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.labels -d MYRIAD -t 0.6

[Olivero__Alberto]

Yes,

I also added the path of the label file as in your example.

No label visible.

I use openvino R5.1 on Myriad X 

Have you seen in the documentation an explanation how to manage this ?

The command I used and the one you suggested appear to me the same you used in the BEST REPLY Mon, 11/26/2018 - 06:25 . Do you see any difference ?

[Lange__Benjamin]

So the commands appear the same to me as well.

I have not found anything in documentation about the topic.

It is a long shot, but you could try the model I used from theTensorFlow Model Zoo

It seems to me that you are using Linux. Have you had a look into the labels file and checked that there is no problem with "end of line characters" and there like? Have you double checked the name of the labels file to be truly "ssd_mobilenet_OLI.labels" ?

You could try debugging the demo code. In the "main.cpp" of the "object_detection_demo_ssd_async" sample code the lines 138 - 143 handle the labels file. You might want to check if the labels are read in correctly by e.g. putting them out on console.

From what I understand if there are read in correctly it is possible that the label container gets cleared if the number of classes mismatch with the labels that were read. For this look at lines 172 -176 in the main.cpp.

Since you are using a different model than I did it might be that the label container gets cleared since the number of classes mismatch.

[Olivero__Alberto]

I SOLVED

many thanks Benjamin, you drive me in the correct direction.

In the file mscoco_label_map.pbtxt there are  90 classes starting from 1=person. From your file (xxx.labels) I removed the last 2 no_label those was class number 91 and 92 In order to have in both files the same number of classes, as for your suggestion. In this way it is working properly.