Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

No performance improvement

SDabh1
Beginner
430 Views

Hello Everyone,

Can anybody help me to understand why I don't see performance improvement in HETERO: FPGA, CPU mode for text detection demo with text_detection_demo{3/4}.xml? In CPU mode, I see performance can go up to ~7-12 FPS but when I tried to use it in HETERO mode It drops down to 5 FPS. I am using the detection and recognition model both in HETERO mode. Even though I use only the detection model in HETERO mode still I see 5fps of throughput.  I am using Mustang SG1 board. Are these numbers relatable or am I missing something here?

 

Thank you.

0 Kudos
3 Replies
Luis_at_Intel
Moderator
430 Views

Hi Dabhade, Sunil,

Thanks for reaching out. May I ask what is the performance when running the demo in FPGA mode? I assume this is the reason why you are using the hetero plugin, as the networks you are using are not fully supported by the FPGA plugin. 

The reason why you may not see a performance improvement is if your FPGA board doesn't support a specific layer, falling back to the CPU may take relatively a longer time affecting the overall performance. I'd suggest to take a look a the Heterogeneous plugin doc for mode information.

Regards,

Luis

0 Kudos
SDabh1
Beginner
430 Views

Hi Luis,

Thanks for the reply.

"May I ask what is the performance when running the demo in FPGA mode?"

       -- I can't run demo in pure FPGA mode as there are some unsupported layer. Hence, I tried it in HETRO mode.

I did some experiment with .xml. I trimmed network at a point where it no longer consists any unsupported layer. And then I ran the network only on FPGA and then performance I see is still same or lower than what I get in CPU mode. After trimming, now xml only consists convolution and clamp layer. I was expecting at least some performance boost here. But I don't see any improvements.

Would like to know why I don't see performance improvement? Is there is something with particular topology?

Thank you and regards,

Sunil

 

0 Kudos
Luis_at_Intel
Moderator
430 Views

Hi Sunil,

Thanks for the information, I am trying to understand what could be happening here. May I ask which topology are you using and how are you converting it to IR format? Also if there is a sample program you are using to test this on your FPGA card. Do you see any performance improvement when running this trimmed network on the CPU?

 

Regards,

Luis

0 Kudos
Reply