Some questions about OpenVINO Post-Training Optimization Tool (POT)

[jacky0327]

I would like to ask some questions about OpenVINO Post-Training Optimization Tool (POT). https://docs.openvinotoolkit.org/latest/pot_README.html

My task is to quantify the YOLOv4-Tiny model of FP32 and FP16 into an INT8 model.

I have referred to similar questions, but I have successfully quantified YOLOv4-Tiny Model (FP32-INT8/FP16-INT8) in OpenVINO 2021.3 and used it normally.
https://community.intel.com/t5/Intel-Distribution-of-OpenVINO/Quantization-8bit-for-yolov4/td-p/1205998

I refer to the parameters of OpenVINO Accuracy Checker Tool to configure the json file needed by POT.
https://docs.openvinotoolkit.org/latest/omz_tools_accuracy_checker_adapters.html

The quantification can be successful, but the mAP after verification is always 0. I want to know where I went wrong.

Please check my attachment, thank you.

My OpenVINO Post-Training Optimization Tool and OpenVINO Accuracy Checker Tool are installed correctly. The version of OpenVINO I use is 2021.3.
The instructions I use are:

Command:

pot -c yolov4-tiny_voc.json --output-dir backup -e

The results that appear are as follows:

INFO:compression.pipeline.pipeline:Evaluation of generated model
INFO:compression.engines.ac_engine:Start inference on the whole dataset
Total dataset size: 26
26 objects processed in 1.023 seconds
INFO:compression.engines.ac_engine:Inference finished
INFO:app.run:detection_accuracy: 0.0

Command:

accuracy_check -c yolov4-tiny_voc.yml -td CPU

The results that appear are as follows:

26 objects processed in 16.514 seconds
09:50:28 accuracy_checker WARNING: /opt/intel/openvino_2021/deployment_tools/open_model_zoo/tools/accuracy_checker/accuracy_checker/metrics/detection.py:201: UserWarning: No detections to compute mAP
warnings.warn("No detections to compute mAP")

map: 0.00%
AP@0.5: 0.00%
AP@0.5:0.05:95: 0.00%

Please help me, why mAP is always 0, please refer to my attachment.

[Zulkifli_Intel]

Hello Peng Chang-Jan

Greetings to you.

Thank you for sharing your model with us. We are investigating the issue and get back to you soon.

Regards,

Zulkifli

[Zulkifli_Intel]

Hello Peng Chang-Jan,

I noticed that this is a duplicate case. Please refer to this thread for the discussion. Therefore, this thread is no longer be monitored by intel.

Regards,

Zulkifli