Yolo v3 mAP is low(OpenVINO R1)

[Atsunori__Sumi]

I measured mAP using this model (OpenVINO: mystic123/tensorflow-yolo-v3). Measurement was performed by remodeling validation_app of OpenVINO. However, the performance value is different from the paper. Is there a problem with this model?
Please let me know if you notice anything.

mAP measurement conditions:
YOLOv3-416@IoU=0.5

OpenVINO validation_app result:
[ INFO ] InferenceEngine:
API version ............ 1.6
Build .................. custom_releases/2019/R1_c9b66a26e4d65bb986bb740e73f58c6e9e84c7c2
[ INFO ] Parsing input parameters
[ INFO ] Loading plugin

API version ............ 1.6 Build .................. 22443 Description ....... MKLDNNPlugin

[ INFO ] Loading network files
[ INFO ] Preparing input blobs
[ INFO ] Batch size is 1
[ INFO ] Device: CPU
[ INFO ] Collecting VOC annotations from /home/dla/sumi/coco/annotations_pascalformat
[ INFO ] 5000 annotations collected
[ INFO ] Starting inference
Progress: [....................] 100.00% done
[ INFO ] Processing output blobs
[ INFO ] Inference report:
Network load time: 112.53ms
Model: mo/yolo_v3.xml
Model Precision: FP32
Batch size: 1
Validation dataset: /home/dla/sumi
Validation approach: Object detection network
[ INFO ] Average infer time (ms): 280.48 (3.56532655 images per second with batch size = 1)
Average precision per class table:

Class AP
1 0.329
2 0.268
3 0.173
4 0.426
5 0.618
6 0.618
7 0.812
8 0.355
9 0.213
10 0.091
11 0.453
12 0.452
13 0.347
14 0.320
15 0.210
16 0.848
17 0.632
18 0.492
19 0.301
20 0.291
21 0.582
22 0.729
23 0.543
24 0.632
25 0.117
26 0.350
27 0.099
28 0.179
29 0.297
30 0.182
31 0.197
32 0.257
33 0.091
34 0.157
35 0.258
36 0.091
37 0.356
38 0.312
39 0.312
40 0.091
41 0.165
42 0.174
43 0.305
44 0.159
45 0.130
46 0.336
47 0.191
48 0.122
49 0.310
50 0.151
51 0.212
52 0.091
53 0.216
54 0.410
55 0.166
56 0.323
57 0.211
58 0.671
59 0.215
60 0.706
61 0.460
62 0.690
63 0.504
64 0.620
65 0.165
66 0.091
67 0.575
68 0.208
69 0.403
70 0.510
71 0.036
72 0.378
73 0.565
74 0.091
75 0.264
76 0.219
77 0.429
78 0.481
79 0.051
80 0.169

Mean Average Precision (mAP): 0.3282

[Shubha_R_Intel]

Dear Atsunori, Sumi,

I already answered your other post regarding the same topic. Hope it helps. If not please let us know.

Thanks,

Shubha