Tiny-yolov3 batch processing

[Chakraborty__Subhasi]

i am using Xeon EG-1275 v6 model server and was trying batch processing for tiny-yolov3. i am getting the following results,

Batch size:: 1 where  fps is ::30

Batch size:: 2 where fps is ::20-22

Batch size::4 where fps is ::11-13

batch size:: 8 where fps is :: 5-7

using GPU as the inferencing platform. I see a steady decrease in fps, when i use batching. It is my expectation to get around 30 fps as according to my knowledge the inferencing must be parallel in nature and should take the same time in every cases.

[nikos1]

> I see a steady decrease in fps

I am not sure how you are measuring this but I am seeing an fps increase.

Batch size:: 1 where  fps is ::30

and then 

Batch size:: 2 where fps is ::20-22 -->> meaning  fps is x2 so you are getting 40-44 fps , correct?

Cheers,

nikos