Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
5763 Discussions

Application Main Thread is hung when loading GPU as a device

Aankesh
Employee
814 Views

When loading GPU as a device  to run the inference, the app main thread is unresponsive for sometime. But this is not the case with CPU or other devices. The same behviour happens when loading complex models like semantic segmentation and object detection. 

 

-Aankesh

0 Kudos
15 Replies
Rizal_Intel
Moderator
802 Views

Hi Aankesh,

 

This is a one-time overhead (order of few seconds) of compiling the OpenCL kernels.

The compilation happens upon loading the network to the GPU plugin and does not affect the inference time.

You can read more about GPU optimization here.

 

If you simply don’t want your main thread from freezing you could run the model asynchronously by running it on another thread.

 

Regards,

Rizal.


Aankesh
Employee
799 Views

hi,

the actual issue is, while the compilation or Running in Async mode, the main threads are getting chocked, even when we are spinning other thread for loading the model. This is seen especially when loading complex models and also durin the GPU device loading.

 

-Aankesh

Rizal_Intel
Moderator
786 Views

Hi Aankesh,


The all thread freezing behavior is not expected.

However, you can use cl_cache method as a workaround to shorten the time for GPU loading.


Regards,

Rizal


Aankesh
Employee
783 Views

Rizal, 

thanks . We wanted to understand why the thread is getting chocked. Wanted to understand if any interrupts are being masked which we are not able to get it through our debugging. 

 

-Aankesh 

Rizal_Intel
Moderator
778 Views

Hi Aankesh,


I did some testing, at it seems at least the keyboards interrupt is registered but deferred (executed later after the GPU plugin finished loading).

How was your debugging result, did the interrupts missing completely (not registered at all) or is it also deferred?


Regards,

Rizal


Aankesh
Employee
761 Views

hi Rizal, 

from our end, all the interrupts are getting blocked. We built a Python GUI app that connects to the openvino. The moment GPU load begins, the entire GUI is hung and all the signals are getting blocked until the gpu load is successful even though we spin a separate thread for the openvino process. 

 

-Aankesh

Aankesh
Employee
706 Views

hi Rizal,

do you have any updates for me on the issue?

 

-Aankesh 

Ray_Lo_Intel
Employee
697 Views

Is there a way to 'cache' the compiled kernel? It is one of the deal breaker for demo as it hang way too long before anything works.

 

Rizal_Intel
Moderator
687 Views

Hi Raymond,

You can use cl_cache method as a workaround to shorten the time for subsequent GPU loading.

Regards,
Rizal

Rizal_Intel
Moderator
685 Views

Hi Aankesh,


We are still investigating this issue, we will inform you on any updates or development on the matter.


Regards,

Rizal


Rizal_Intel
Moderator
582 Views

Hi Aankesh,


The investigation shows that this script hits the well-known Python GIL problem, which is described in links below along with some clues on how to workaround it:

https://realpython.com/python-gil/

https://hackernoon.com/has-the-python-gil-been-slain-9440d28fa93d?gi=a9cc9f35638c 

So the problem reported could not be considered as an OpenVINO bug.


Regards,

Rizal


Rizal_Intel
Moderator
552 Views

Hi Aankesh,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


Aankesh
Employee
544 Views

hi Rizal,

This issue is happening in c++ as well. Please validate from your end also.

 

-Aankesh 

Rizal_Intel
Moderator
519 Views

Hi Aankesh,


Would you kindly provide more details for the issue in C++?

Please provide some replication code if possible for me to verify this issue on C++.


Regards,

Rizal


Rizal_Intel
Moderator
486 Views

Hi Aankesh,


Thank you for your question.

Intel will no longer monitor this thread . If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


Reply