Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

about openVINO-caffe SSD inference time

center__Chen
Beginner
905 Views

A、The following points are the conditions I have encountered :

     1、When I use 「object_detection_sample_ssd.exe」to do inference test , I get a bad FPS ,

          This inference time is much worse than the paper results

          (1_1) caffe_SSD_VGG_512x512 -> FPS only about 1-2

          (1_2) caffe_SSD_VGG_300x300 -> FPS only about 5-6

     2、Undertake , I tried to test with 「openvino and opencv api」, but FPS still get the bad result

     3、And I also use 「benchmark_app」 to test FPS value , but FPS still get the same vaule 

 

B、The following is my TEST ENVIRONMENT:

      OS : win10

      CPU : intel(R) xeon(R) cpu e3-1505m v6 @ 3.00ghz 

      VPU : Mustang-V100-MX8

      Model : convert by myself (use mo_caffe.py)

                   and model is download following URL 

                   https://github.com/weiliu89/caffe/tree/ssd

 

C、So what I want to know is :

      1、Is this FPS normal on my TEST ENVIRONMENT ? because this FPS result vaule is lower than paper 

           if not , How can I improve and correct ?

 

looking forward to your reply

 

--

Center 
 

0 Kudos
10 Replies
Shubha_R_Intel
Employee
905 Views

Dear center, Chen,

Your observations are valid. Are you using OpenVino 2019R2 ? Do you get the same bad results for SSD on the latest release (which is OpenVino 2019R2) ? If these latest results are on OpenVino 2019R2, then I will file a bug on your behalf.

Please let me know,

Shubha

0 Kudos
center__Chen
Beginner
905 Views

Hi Shubha,

 

Yes , I used R2 (install in this week)

through my test and fine some information , 

A、

if use Caffe-「MobileNet」-SSD , the FPS  about falling 10-20

but if I use Caffe-「VGG」-SSD , the FPS get so bad (as the title shows)

B、

Because the project need to used Caffe-VGG-SSD to do performance NOT MobileNet

 

Can you help me to reply this soulution ?

thanks

 

Center 

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, Chen

Precisely which model is it from here ?

Thanks,

Shubha

0 Kudos
center__Chen
Beginner
905 Views

Hi Shubha,

My model is download by following URL

https://github.com/weiliu89/caffe/tree/ssd

 

This URL is for VGGNet_VOC0712_SSD_300x300

https://drive.google.com/file/d/0BzKzrI_SkD1_WVVTSmQxU0dVRzA/view

And this is for VGGNet_VOC0712_SSD_512x512

https://drive.google.com/file/d/0BzKzrI_SkD1_ZDIxVHBEcUNBb2s/view

 

Center

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, Chen,

Thank you ! I will investigate this and report back on this forum.

Shubha

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, ,

I am sorry but I'm not able to reproduce your terrible performance results from SSD Caffe models, using OpenVino 2019R2.01. For instance I just  tried bvlc_reference_rcnn_ilsvrc13 from  https://github.com/weiliu89/caffe/tree/ssd/models.

My MO command is :

python "c:\Program Files (x86)\intelswtools\openvino_2019.2.275\deployment_tools\model_optimizer\mo_caffe.py" --input_model bvlc_reference_rcnn_ilsvrc13.caffemodel --input_proto deploy.prototxt.txt

Then I ran benchmark_app and the results were considerably better than yours, and I'm using a plain Windows 10 laptop (trust me, nothing special):

Here are my results:

benchmark_app.exe -m c:\users\sdramani\Downloads\bvlc_reference_rcnn_ilsvrc13.xml -i c:\users\sdramani\Downloads\pics\horse1.jpg
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[ INFO ] Files were added: 1
[ INFO ]     c:\users\sdramani\Downloads\pics\horse1.jpg
[Step 2/11] Loading Inference Engine
[ INFO ] InferenceEngine:
        API version ............ 2.0
        Build .................. 27579
        Description ....... API
[ INFO ] Device info:
        CPU
        MKLDNNPlugin version ......... 2.0
        Build ........... 27579

[Step 3/11] Reading the Intermediate Representation network
[ INFO ] Loading network files
[Step 4/11] Resizing network to match image sizes and given batch
[ INFO ] Network batch size: 10, precision: FP32
[Step 5/11] Configuring input of the model
[Step 6/11] Setting device configuration
[Step 7/11] Loading the model to the device
[Step 8/11] Setting optimal runtime parameters
[Step 9/11] Creating infer requests and filling input blobs with images
[ INFO ] Network input 'data' precision U8, dimensions (NCHW): 10 3 227 227
[ WARNING ] Some image input files will be duplicated: 40 files are required but only 1 are provided
[ INFO ] Infer Request 0 filling
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Infer Request 1 filling
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Infer Request 2 filling
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Infer Request 3 filling
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[ INFO ] Prepare image c:\users\sdramani\Downloads\pics\horse1.jpg
[ WARNING ] Image is resized from (1000, 563) to (227, 227)
[Step 10/11] Measuring performance (Start inference asyncronously, 4 inference requests using 4 streams for CPU, limits: 60000 ms duration)

[Step 11/11] Dumping statistics report
[ INFO ] Statistics collecting was not requested. No reports are dumped.
Count:      540 iterations
Duration:   60774.4 ms
Latency:    444.875 ms
Throughput: 88.8532 FPS

I didn't try your custom trained models because really there's no point. The basic pre-trained models from here which you referred me have good performance.

Sincerely,

Shubha

 

0 Kudos
center__Chen
Beginner
905 Views

Hi Shubha:

thanks for reply

I have some question to asked:

 

(A) I have used benchmark to do the test before.,

「benchmark_app」 show FPS = 35 ( -d HDDL) 

BUT when I use 「object_detection_sample_ssd.exe」 (-d HDDL) 

FPS only 5-6

 

(B) why are you not using my provide model to test ? Just use this instead of other models, this is the model provided by the SSD author, I have not made any changes at all

my model is dowload from following URL , and click 「07+12: SSD300*, SSD512*」

https://github.com/weiliu89/caffe/blob/ssd/README.md#models

 

 

(C) I need your test data , NOT benchmark_app show FPS,

 because 「benchmark_app 」FPS Value is  much worse than 「object_detection_sample_ssd.exe」

 

Center

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, Chen

I am definitely sorry about missing your objectives. Certainly i will try again to run tests according to your specific asks. Please stay tuned on this forum and I will post my findings.

Sincerely,

Shubha

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, Chen,

I tried your model VGGNet_VOC0712_SSD_300x300.tar just now with both the C++ object_detection_demo_ssd_async.exe, the object_detection_demo_ssd.exe and also the Python object_detection_demo_ssd_async.py on various JPGs, and the detection happened almost instantaneously. I also tried object_detection_demo_ssd_async.exe on *.mp4 videos and didn't notice slow performance (although detection didn't happen with the videos). I tried all this on OpenVino 2019R3 (which you don't have). If there are performance issues with SSD models, I believe that the issues have been fixed by R3.

Hope it helps,

thanks,

Shubha

0 Kudos
Shubha_R_Intel
Employee
905 Views

Dear center, Chen,

I would like to apologize to you though. Your system is entirely different than mine. Just because OpenVino SSD performed well on my laptop doesn't mean that your experience is the same. So, sorry for being presumptuous about your situation. That said, we are aware of performance issues on mobilenet type models (which have SSD flavors).  Looks like yours are not mobilenet, they are VggNet. One suggestion for you - please try INT8 calibration on those models. Hopefully you will achieve better performance with an acceptable loss of accuracy.

Hope it helps,

Shubha

0 Kudos
Reply