Inference on GPU

[momo93]

I'm doing inference on my GPU, it's works fine, I was comparing RAM usage between CPU and GPU for the same frame rate, GPU consume RAM (twice ) more than CPU. do you have an explanation for this, and how to resolve it, im doing some Embedded application and I can not  offer this amount of ram, for my app.

another problem when deploying, I noticed that  the libs installed for open cl and intel compute run time are huge, can we reduce them ? 

I'm working on linux 

thanks in advance  

[Hairul_Intel]

Hi momo,

Thank you for reaching out to us.

I've validated from my end by running the Object Detection Python Demo using face-detection-adas-0001 model on CPU and GPU. The results are shown below:

CPU:

GPU:

The results shows that the difference in memory used between CPU and GPU is around 51MB only, which is not a really significant amount.

On the other hand, would you please share with us which demo/sample gives you a huge difference in RAM consumption?

For your second question, unfortunately, reducing the driver components size is not an option as Intel® Graphics Compute Runtime for OpenCL™ driver components are required to use the GPU plugin and write custom layers for Intel® Integrated Graphics in OpenVINO Toolkit.

Regards,

Hairul

[momo93]

i was testing object detection demo on c++, however I think my issues is related to the version of compute rutime intel used, Im using the last release 

[Hairul_Intel]

Hi momo,

I've validated the Object Detection C++ Demo using face-detection-adas-0001 model on CPU and GPU in Ubuntu 20.04 LTS. Here are the RAM usage results:

CPU (used 1360MB RAM) :

GPU (used 1458MB RAM) :

The idle RAM usage is shown below (used 1173MB):

For your information, here is my Intel® Graphics Compute Runtime for OpenCL™ version (20.35.17767) and GPU specification:

From my end, the GPU uses more RAM as compared to CPU but not double the amount. The CPU and GPU RAM usage are as follows:

• CPU (1360MB used - 1173MB used) = 187MB
• GPU (1458MB used - 1173MB used) = 285MB
• GPU - CPU = 98MB

Could you please provide the following information for further investigation:

1. GPU device specification (Run the Hello Query Device Python Sample to find your device information).
2. Intel® Graphics Compute Runtime for OpenCL™ driver version.
3. Exact value for CPU and GPU RAM usage when you ran the demo.

Regards,

Hairul

[Hairul_Intel]

Hi momo,

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.

Regards,

Hairul