understanding model optimizer

[pkhan10]

Hello..
I am using openvino model optimizer to convert couple of different model in tensorflow model zoo..
but in inference mode  the performance(FPS) differs a lot 

for ssd_mobilenet_v2_coco i get fps of 40-45 in normal (infer) mode and 150-200 in async mode

but for ssd_resnet_50_fpn_coco  i get fps 1.2-1.5 in  normal(infer) model and 2-5 in async mode

as you can in in tf model zoo...fps is just reduce half on gpu..
so why such drastic change in cpu?
please help me understand how model optimizer works

[Shubha_R_Intel]

Dear khandelwal, prateek

ssd_resnet_50_fpn_coco only getting 1.2-1.5  fps in  normal(infer) model and 2-5 fps in async mode versus ssd_mobilenet_v2_coco getting fps of 40-45 in normal (infer) mode and 150-200 in async mode has nothing to do with Model Optimizer. Model Optimizer is a complex piece of pure Python code that takes in a graph (Deep Learning model) and converts it to IR - Model Optimizer is basically a compiler. True, some optimization does occur in the Model Optimizer phase but the lion-share of optimization actually occurs when the hardware plugin loads the model.

Are you only measuring on CPU and GPU ? Are you saying that the FPS is much less on GPU ? I'm not sure I understand your question. 

Thanks,

Shubha

[pkhan10]

All performance are in CPU mode only..
But I am talking about performance observed...
decrement in FPS was way less when compare in nvidia gpu as compare to intel CPU

Is there's performance table, as which model performance best on intel hardware?

[Shubha_R_Intel]

Dear khandelwal, prateek,

We don't have performance numbers available for OpenVino unfortunately. But we are working on this as many customers have asked for it. Keep in mind, many assumptions go into bench-marking numbers. Unless you compare an Intel CPU system built exactly like the Nvidia system (memory wise, CPU wise, OS kernel wise, hard disk size-wise, and many-other-things-wise), it's improper to draw conclusions or even compare the two.

Hope it helps,

Thanks,

Shubha

[pkhan10]

Hello shubha..
It would be great if you can suggest what architecture are good to use to get decent performance..
I tried FasterRCNN, retinanet , ssd  FPN all gave less than 2 FPS..
while ssd_mobilenet, squeezenet  gave atleast 40 FPS..

former ones have very goof mAP on low resolution object..
It would be great if you can guide..
which higher accuracy model I can use for having better results

[Shubha_R_Intel]

Dear khandelwal, prateek,

My recommendation would be to use the benchmark_app , perform some experiments, and draw your own conclusions.

Hope it helps,

Thanks,

Shubha