Community
cancel
Showing results for 
Search instead for 
Did you mean: 
171 Views

Hetero Plugin doesn't increase performance

Jump to solution

Hi I'm using OpenVINO 2019_R3, I created my application in Python using the Inference Engine and did some performance tests. In my application I process an image and therefore I decided to test with 600 images to have the average value. When I use the FP32 model on a CPU I have obtained almost the same results as the FP16 on CPU (10 FPS), the use of the FP32 on a GPU increases the performance (20 FPS) which increase with the FP16 model (34 FPS) . I checked the documentation and saw the Hetero plugin to use all the devices I have at my best. Why does my performance decrease if I use HETERO: GPU, CPU (22 FPS) or HETERO: CPU, GPU (11 FPS) with FP16 model?

If I use FP32, the results are: 10 FPS (HETERO:CPU,GPU) and  18 FPS (HETERO:GPU,CPU).

Is it possible that in my application and with my devices the best result is obtained only with the GPU?

Best Regards,

Cristian.

0 Kudos
1 Solution
SuryaPSC_Intel
Employee
171 Views

Hi Cristian,

HETRO mode uses the fallback policy to perform inference on multiple devices. However it does not ensure enhanced performance as transmitting of data from one part of network to another part in heterogeneous mode may take relatively much time.

So, if the network is supported in a single device it is recommended to use that device.

Best Regards,

Surya

View solution in original post

4 Replies
SuryaPSC_Intel
Employee
171 Views

Hi Cristian,

Can you please confirm if the result (HETRO: GPU, CPU (22 FPS) or HETERO: CPU, GPU (11 FPS))  is with FP32 or FP16 model?

Best Regards,

Surya

171 Views

Hi Surya,

The results are with FP16 model. If I use FP32, the results are: 10 FPS (HETERO:CPU,GPU) and  18 FPS (HETERO:GPU,CPU).

Best Regards,
Cristian

171 Views

Any advice?

Thanks. 

 

SuryaPSC_Intel
Employee
172 Views

Hi Cristian,

HETRO mode uses the fallback policy to perform inference on multiple devices. However it does not ensure enhanced performance as transmitting of data from one part of network to another part in heterogeneous mode may take relatively much time.

So, if the network is supported in a single device it is recommended to use that device.

Best Regards,

Surya

View solution in original post

Reply