Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

Tiny-yolov3 batch processing

Chakraborty__Subhasi
240 Views

i am using Xeon EG-1275 v6 model server and was trying batch processing for tiny-yolov3. i am getting the following results,

Batch size:: 1 where  fps is ::30

Batch size:: 2 where fps is ::20-22

Batch size::4 where fps is ::11-13

batch size:: 8 where fps is :: 5-7

using GPU as the inferencing platform. I see a steady decrease in fps, when i use batching. It is my expectation to get around 30 fps as according to my knowledge the inferencing must be parallel in nature and should take the same time in every cases.

0 Kudos
1 Reply
nikos1
Valued Contributor I
240 Views

> I see a steady decrease in fps

I am not sure how you are measuring this but I am seeing an fps increase.

Batch size:: 1 where  fps is ::30

and then 

Batch size:: 2 where fps is ::20-22 -->> meaning  fps is x2 so you are getting 40-44 fps , correct?

Cheers,

nikos

0 Kudos
Reply