Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Aankesh
Employee
492 Views

Application Main Thread is hung when loading GPU as a device

When loading GPU as a device  to run the inference, the app main thread is unresponsive for sometime. But this is not the case with CPU or other devices. The same behviour happens when loading complex models like semantic segmentation and object detection. 

 

-Aankesh

0 Kudos
15 Replies
Rizal_Intel
Moderator
480 Views

Hi Aankesh,

 

This is a one-time overhead (order of few seconds) of compiling the OpenCL kernels.

The compilation happens upon loading the network to the GPU plugin and does not affect the inference time.

You can read more about GPU optimization here.

 

If you simply don’t want your main thread from freezing you could run the model asynchronously by running it on another thread.

 

Regards,

Rizal.


Aankesh
Employee
477 Views

hi,

the actual issue is, while the compilation or Running in Async mode, the main threads are getting chocked, even when we are spinning other thread for loading the model. This is seen especially when loading complex models and also durin the GPU device loading.

 

-Aankesh

Rizal_Intel
Moderator
464 Views

Hi Aankesh,


The all thread freezing behavior is not expected.

However, you can use cl_cache method as a workaround to shorten the time for GPU loading.


Regards,

Rizal


Aankesh
Employee
461 Views

Rizal, 

thanks . We wanted to understand why the thread is getting chocked. Wanted to understand if any interrupts are being masked which we are not able to get it through our debugging. 

 

-Aankesh 

Rizal_Intel
Moderator
456 Views

Hi Aankesh,


I did some testing, at it seems at least the keyboards interrupt is registered but deferred (executed later after the GPU plugin finished loading).

How was your debugging result, did the interrupts missing completely (not registered at all) or is it also deferred?


Regards,

Rizal


Aankesh
Employee
439 Views

hi Rizal, 

from our end, all the interrupts are getting blocked. We built a Python GUI app that connects to the openvino. The moment GPU load begins, the entire GUI is hung and all the signals are getting blocked until the gpu load is successful even though we spin a separate thread for the openvino process. 

 

-Aankesh

Aankesh
Employee
384 Views

hi Rizal,

do you have any updates for me on the issue?

 

-Aankesh 

Raymond
Employee
375 Views

Is there a way to 'cache' the compiled kernel? It is one of the deal breaker for demo as it hang way too long before anything works.

 

Rizal_Intel
Moderator
365 Views

Hi Raymond,

You can use cl_cache method as a workaround to shorten the time for subsequent GPU loading.

Regards,
Rizal

Rizal_Intel
Moderator
363 Views

Hi Aankesh,


We are still investigating this issue, we will inform you on any updates or development on the matter.


Regards,

Rizal


Rizal_Intel
Moderator
260 Views

Hi Aankesh,


The investigation shows that this script hits the well-known Python GIL problem, which is described in links below along with some clues on how to workaround it:

https://realpython.com/python-gil/

https://hackernoon.com/has-the-python-gil-been-slain-9440d28fa93d?gi=a9cc9f35638c 

So the problem reported could not be considered as an OpenVINO bug.


Regards,

Rizal


Rizal_Intel
Moderator
231 Views

Hi Aankesh,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


Aankesh
Employee
222 Views

hi Rizal,

This issue is happening in c++ as well. Please validate from your end also.

 

-Aankesh 

Rizal_Intel
Moderator
197 Views

Hi Aankesh,


Would you kindly provide more details for the issue in C++?

Please provide some replication code if possible for me to verify this issue on C++.


Regards,

Rizal


Rizal_Intel
Moderator
164 Views

Hi Aankesh,


Thank you for your question.

Intel will no longer monitor this thread . If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


Reply