Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6392 Discussions

Labels missing in object_detection_demo_ssd_async demo

Lange__Benjamin
2,615 Views

Hello everyone,

this is my software environment:
Windows 10
CMake 3.1
Python 3.5
Visual Studio 2017
OpenCV 4.0
Intel C++ Compiler 2017 Redistributable package for Windows
Neural Compute Stick (Myriad 2)

I converted a model from the Tensorflow detection model zoo (https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md) namely "ssd_mobilenet_v1_0_75_depth_300x300_coco14_sync_2018_07_03".

The demo I ran was "object_detection_demo_ssd_async demo". The demo worked fine but the labels were missing. Reviewing the source code it seems to me that I have to provide a labels file with the "-m" option. Providing a labels file is mentioned somewhere in the the docs for the demo (https://software.intel.com/en-us/articles/OpenVINO-IE-Samples#object-detection-SSD-showcase) as a side note. But an explicit example is missing or was overlooked (?).

The labels file is a simply a file where each label has its own line. I named it with the ending ".labels" since according to the source code it is the ending the program is looking for.

The command I issue looks like this:

Intel\OpenVINO\inference_engine_samples_2017\intel64\Release\object_detection_demo_ssd_async.exe -i cam -m F:\ssd_mobilenet_v1_0-75_300x300_coco14_ir_fp16\frozen_inference_graph.xml F:\ssd_mobilenet_v1_0-75_300x300_coco14_ir_fp16\coco.labels -d MYRIAD

Does anyone have a clue what might be going wrong here?

Best regards,

B. Lange

 

0 Kudos
1 Solution
Lange__Benjamin
2,615 Views

Ok found my error, I did not read the code properly.

The name of the labels file needs to be named after the input model e.g. "frozen_inference_graph.labels".

Now everything works fine, case closed.

View solution in original post

16 Replies
Lange__Benjamin
2,616 Views

Ok found my error, I did not read the code properly.

The name of the labels file needs to be named after the input model e.g. "frozen_inference_graph.labels".

Now everything works fine, case closed.

Altmaier__Richard
2,615 Views

Hi, I am new to Intel's OpenVINO distribution and tensorflow models.  I have a simple question, where is the labels file to be found, which corresponds to any given tensorflow model, such as the ones at https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

Thanks! 

0 Kudos
SC_Huang
Beginner
2,615 Views

Altmaier, Richard wrote:

Hi, I am new to Intel's OpenVINO distribution and tensorflow models.  I have a simple question, where is the labels file to be found, which corresponds to any given tensorflow model, such as the ones at https://github.com/tensorflow/models/blob/master/research/object_detecti...

Thanks! 

I have same question, even i've found each object of the IDs like below, I still cannot figure out the data structure of the labels file.

https://tech.amikelive.com/node-718/what-object-categories-labels-are-in-coco-dataset/

 

If someone can supply the label file or simply describe the structure of the lables file will be great

 

0 Kudos
nikos1
Valued Contributor I
2,615 Views

Hello Huang,

The .labels file should have the same name as .bin and .xml for the sample applications in OpenVino to read.

For example: yolo_v3.bin, yolo_v3.xml , yolo_v3.labels all in the same folder.

The .labels files is a simple text file with each class name in every line like for example

kite
baseball bat
baseball glove
skateboard
surfboard
tennis racket
bottle
wine glass
cup
fork
knife
spoon
bowl
banana
apple
sandwich
orange
broccoli
carrot
hot dog
pizza
donut
. . . 

Cheers,

Nikos

0 Kudos
Severine_H_Intel
Employee
2,615 Views

Dear all, 

in the labels file, you should withdraw the whitespace, otherwise, it considers: wine glass as 2 labels. You can replace it with an underscore for example. 

Best, 

Severine

0 Kudos
Lange__Benjamin
2,615 Views

Not all labels are used!

My label file for e.g. ssd_mobilenet_v1_0.75_depth_coco is attached to this post.

You have to remove the "doc" ending. I added it to be able to upload the file.

Do not remove any of the empty lines in the document! Otherwise you will get wrong label IDs.
You can derive the IDs from this file:

https://github.com/tensorflow/models/blob/master/research/object_detection/data/mscoco_label_map.pbtxt

It tells you which IDs are actually used and which are not.

How do you know which label map is used?

This information is written in the config, which can be found here: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/ssd_mobilenet_v1_0.75_depth_300x300_coco14_sync.config

You have to look at the very bottom of the config where it says:

"label_map_path: "PATH_TO_BE_CONFIGURED/mscoco_label_map.pbtxt"

It took me a while to figure this out, because I could not find a source explaining this.

0 Kudos
nikos1
Valued Contributor I
2,615 Views

Hello Benjamin,

I am not sure if I understand your question correctly but please note as Severine mentioned above : 

in the labels file, you should withdraw the whitespace, otherwise, it considers: wine glass as 2 labels. You can replace it with an underscore for example. 

Your attached frozen_inference_graph.labels.doc is not a good OpenVino labels file.

For example 

stop sign

should become something like

stop_sign

unless of course you changed the sample source code.

Also I am not sure how the current code will deal with the empty lines. Better to replace with an actual label like "empty" or :"no_label"

cheers,

nikos

0 Kudos
Lange__Benjamin
2,615 Views

I did not experience any problems with whitespaces. The empy lines just worked fine as well.

My problem was mainly to find the labels file and finding out which IDs were actually used.

I attached the changed file according to Nikos suggestions.

I cannot find the option to delete the previously uploaded file though.

0 Kudos
tay__lenelle
Beginner
2,615 Views

Hi,

I would like to understand that after converting my tensorflow model to .xml and .bin files, i have successfully ran the object detection. However, there were no labels appearing with the bounding box. I would like to seek help in getting the labels please thank you.

0 Kudos
Olivero__Alberto
Beginner
2,615 Views

Lange, Benjamin wrote:

I did not experience any problems with whitespaces. The empy lines just worked fine as well.

My problem was mainly to find the labels file and finding out which IDs were actually used.

I attached the changed file according to Nikos suggestions.

I cannot find the option to delete the previously uploaded file though.

Hi Benjamin, I'm using "ssd_mobilenet_v1_coco_2018_01_28" and the inference command as for your suggestion is

./object_detection_demo_ssd_async -i cam -m ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.xml ssd_mobilenet_OLI.labels -d MYRIAD -t 0.6

but I still see label #1, laber 36# and no names. Am I wrongly applying your suggestion ?

0 Kudos
Lange__Benjamin
2,615 Views

Hi Olivero,

are you using the label file I uploaded in the post you cited?

The command you used will show you no labels and it will not throw an error if you actually did not provide a label file.

Would you upload your label file so that I can have a look?

Kind regards

Benjamin

0 Kudos
Olivero__Alberto
Beginner
2,615 Views

Hi Benjamin,

  • I used your label file (the one where you added underscore in the post Tue, 02/05/2019 - 01:01).
  • I just changed the file name to use the same name of my model as for blog suggest.
  • I put it in the same folder of SSD_mobilenet model: ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/

 

BR

Alberto

0 Kudos
Lange__Benjamin
2,615 Views

Have you tried the command providing the path to the labels file like this:

./object_detection_demo_ssd_async -i cam -m ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.xml ~/Scaricati/ssd_mobilenet_v1_coco_2018_01_28/ssd_mobilenet_OLI.labels -d MYRIAD -t 0.6

 

0 Kudos
Olivero__Alberto
Beginner
2,615 Views

Yes,

I also added the path of the label file as in your example.

No label visible.

I use openvino R5.1 on Myriad X 

Have you seen in the documentation an explanation how to manage this ?

The command I used and the one you suggested appear to me the same you used in the BEST REPLY Mon, 11/26/2018 - 06:25 . Do you see any difference ?

0 Kudos
Lange__Benjamin
2,615 Views

So the commands appear the same to me as well.

I have not found anything in documentation about the topic.

It is a long shot, but you could try the model I used from theTensorFlow Model Zoo

It seems to me that you are using Linux. Have you had a look into the labels file and checked that there is no problem with "end of line characters" and there like? Have you double checked the name of the labels file to be truly "ssd_mobilenet_OLI.labels" ?

You could try debugging the demo code. In the "main.cpp" of the "object_detection_demo_ssd_async" sample code the lines 138 - 143 handle the labels file. You might want to check if the labels are read in correctly by e.g. putting them out on console.

From what I understand if there are read in correctly it is possible that the label container gets cleared if the number of classes mismatch with the labels that were read. For this look at lines 172 -176 in the main.cpp.

Since you are using a different model than I did it might be that the label container gets cleared since the number of classes mismatch.

0 Kudos
Olivero__Alberto
Beginner
2,615 Views

I SOLVED

many thanks Benjamin, you drive me in the correct direction.

In the file mscoco_label_map.pbtxt there are  90 classes starting from 1=person. From your file (xxx.labels) I removed the last 2 no_label those was class number 91 and 92 In order to have in both files the same number of classes, as for your suggestion. In this way it is working properly.

0 Kudos
Reply