OpenVINO DL Workbench Accuracy Check for yolo

[Fai]

I tried accuracy check for yolo-v2-tiny-tf on DL workbench, but it occured warning.
I used omz model, and dataset is coco.
How to accuracy check for yolo model on DL Workbench?

[Iffa_Intel]

Greetings,

If possible can you share your working model?

Plus with the commands you had used and the errors snippets.

Sincerely,

Iffa

[Fai]

Hi Iffa,

Thank you for replying me.

My working model imported from Open Model Zoo, yolo-v2-tiny-tf.

The details of that model are described as follows.

"YOLO v2 Tiny is a real-time object detection model implemented with Keras* from this repository <https://github.com/david8862/keras-YOLOv3-model-set> and converted to TensorFlow* framework. This model was pretrained on COCO* dataset with 80 classes."

I run Accuracy Check from DL workbench, the Configure Accuracy is set as follows.

• Basic / Advanced : Basic
• Usage : Object Detection
• Model Type : Tiny Yolo V2
• Resize Type : Auto
• Prediction Boxes : Resize Boxes
• Metric : COCO Precision
• Max Detections : 100
• Separete Background Class : No
• Predictions are mapped to : 80 COCO classes

The Warning messages are as follows.

accuracy_checker WARNING:/opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseinputChannels.py:134:

DeprecationWarning: invalid escape sequence \

accuracy_checker WARNING:/opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseinputChannels.py:194:

DeprecationWarning: invalid escape sequence \

accuracy_checker WARNING:/opt/intel/openvino/deployment_tools/model_optimizer/extensions/back/ReverseinputChannels.py:299:

DeprecationWarning: invalid escape sequence \

Regards,

Fai

[VladimirG]

Greetings @Fai 

It seems like in the case of yolo-v2-tiny-tf model, the pre-defined accuracy config downloaded along with the model is not quite sufficient for measurements. In order to circumvent this issue, you can try the following:
 

1. Go to the Accuracy measurement page, and select the Advanced configuration option - it should already have a predefined configuration pulled from the OMZ
2. In the adapter parameter under the launchers section, add the following argument: "classes: 80"

This should hopefully resolve the issue and allow you to measure accuracy on this model.

[Vladimir_Dudnik]

@Fai you may also review accuracy checker configuration file provided with yolo-v2-tiny-tf OMZ model to see parameters used in accuracy validation at Open Model Zoo side.

[Alexander_D_Intel1]

Hello @Fai !

Thank you for asking.

@VladimirG  has provided the full answer on your question. 

If you have any additional questions or proposals for using YOLO models in DL Workbench, please ask them at the official DL Workbench Discussion Forum: https://github.com/openvinotoolkit/workbench_feedback/discussions.

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Sincerely,

Iffa