do infernce on GPU cost 57ms but 30ms on CPU - Intel Community

Intel® Distribution of OpenVINO™ Toolkit

Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.

do infernce on GPU cost 57ms but 30ms on CPU

453 Views

I have a model ,it only cost 30ms on CPU, but 57ms on GPU. the time cost is average value by 1000 times.

how can I reduce execution time on GPU?

Link Copied

2 Replies

453 Views

there are five "Interp" layers in the model， is this the main reson that doing inference on GPU device cost much more time?

Copy link

453 Views

Does anyone know the reason？

Copy link

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.