Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.

Inference on GPU

momo93
Novice
471 Views

I'm doing inference on my GPU, it's works fine, I was comparing RAM usage between CPU and GPU for the same frame rate, GPU consume RAM (twice ) more than CPU. do you have an explanation for this, and how to resolve it, im doing some Embedded application and I can not  offer this amount of ram, for my app.

another problem when deploying, I noticed that  the libs installed for open cl and intel compute run time are huge, can we reduce them ? 

 

I'm working on linux 

thanks in advance  

 

 

0 Kudos
4 Replies
Hairul_Intel
Moderator
409 Views

Hi momo,

Thank you for reaching out to us.

 

I've validated from my end by running the Object Detection Python Demo using face-detection-adas-0001 model on CPU and GPU. The results are shown below:

 

CPU:

CPU.png

 

GPU:

GPU.png

 

 

The results shows that the difference in memory used between CPU and GPU is around 51MB only, which is not a really significant amount.

 

On the other hand, would you please share with us which demo/sample gives you a huge difference in RAM consumption?

 

For your second question, unfortunately, reducing the driver components size is not an option as Intel® Graphics Compute Runtime for OpenCL™ driver components are required to use the GPU plugin and write custom layers for Intel® Integrated Graphics in OpenVINO Toolkit.

 

 

Regards,

Hairul

 

momo93
Novice
343 Views

i was testing object detection demo on c++, however I think my issues is related to the version of compute rutime intel used, Im using the last release 

Hairul_Intel
Moderator
305 Views

Hi momo,

I've validated the Object Detection C++ Demo using face-detection-adas-0001 model on CPU and GPU in Ubuntu 20.04 LTS. Here are the RAM usage results:

 

CPU (used 1360MB RAM) :

cpu.jpg

 

GPU (used 1458MB RAM) :

gpu.jpg

 

The idle RAM usage is shown below (used 1173MB):

idle.jpg

 

 

For your information, here is my Intel® Graphics Compute Runtime for OpenCL™ version (20.35.17767) and GPU specification:

driver.jpg

device.png

 

 

From my end, the GPU uses more RAM as compared to CPU but not double the amount. The CPU and GPU RAM usage are as follows:

  • CPU (1360MB used - 1173MB used) = 187MB
  • GPU (1458MB used - 1173MB used) = 285MB
  • GPU - CPU = 98MB

 

Could you please provide the following information for further investigation:

  1. GPU device specification (Run the Hello Query Device Python Sample to find your device information).
  2. Intel® Graphics Compute Runtime for OpenCL™ driver version.
  3. Exact value for CPU and GPU RAM usage when you ran the demo.

 

 

Regards,

Hairul

 

Hairul_Intel
Moderator
242 Views

Hi momo,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

 

 

Regards,

Hairul


Reply