Intel® Distribution of OpenVINO™ Toolkit
Community assistance about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all aspects of computer vision-related on Intel® platforms.
6411 Discussions

Application Main Thread is hung when loading GPU as a device

Aankesh
Employee
2,083 Views

When loading GPU as a device  to run the inference, the app main thread is unresponsive for sometime. But this is not the case with CPU or other devices. The same behviour happens when loading complex models like semantic segmentation and object detection. 

 

-Aankesh

0 Kudos
15 Replies
Rizal_Intel
Moderator
2,071 Views

Hi Aankesh,

 

This is a one-time overhead (order of few seconds) of compiling the OpenCL kernels.

The compilation happens upon loading the network to the GPU plugin and does not affect the inference time.

You can read more about GPU optimization here.

 

If you simply don’t want your main thread from freezing you could run the model asynchronously by running it on another thread.

 

Regards,

Rizal.


0 Kudos
Aankesh
Employee
2,068 Views

hi,

the actual issue is, while the compilation or Running in Async mode, the main threads are getting chocked, even when we are spinning other thread for loading the model. This is seen especially when loading complex models and also durin the GPU device loading.

 

-Aankesh

0 Kudos
Rizal_Intel
Moderator
2,055 Views

Hi Aankesh,


The all thread freezing behavior is not expected.

However, you can use cl_cache method as a workaround to shorten the time for GPU loading.


Regards,

Rizal


0 Kudos
Aankesh
Employee
2,052 Views

Rizal, 

thanks . We wanted to understand why the thread is getting chocked. Wanted to understand if any interrupts are being masked which we are not able to get it through our debugging. 

 

-Aankesh 

0 Kudos
Rizal_Intel
Moderator
2,047 Views

Hi Aankesh,


I did some testing, at it seems at least the keyboards interrupt is registered but deferred (executed later after the GPU plugin finished loading).

How was your debugging result, did the interrupts missing completely (not registered at all) or is it also deferred?


Regards,

Rizal


0 Kudos
Aankesh
Employee
2,030 Views

hi Rizal, 

from our end, all the interrupts are getting blocked. We built a Python GUI app that connects to the openvino. The moment GPU load begins, the entire GUI is hung and all the signals are getting blocked until the gpu load is successful even though we spin a separate thread for the openvino process. 

 

-Aankesh

0 Kudos
Aankesh
Employee
1,975 Views

hi Rizal,

do you have any updates for me on the issue?

 

-Aankesh 

0 Kudos
Ray_Lo_Intel
Employee
1,966 Views

Is there a way to 'cache' the compiled kernel? It is one of the deal breaker for demo as it hang way too long before anything works.

 

0 Kudos
Rizal_Intel
Moderator
1,956 Views

Hi Raymond,

You can use cl_cache method as a workaround to shorten the time for subsequent GPU loading.

Regards,
Rizal

0 Kudos
Rizal_Intel
Moderator
1,954 Views

Hi Aankesh,


We are still investigating this issue, we will inform you on any updates or development on the matter.


Regards,

Rizal


0 Kudos
Rizal_Intel
Moderator
1,851 Views

Hi Aankesh,


The investigation shows that this script hits the well-known Python GIL problem, which is described in links below along with some clues on how to workaround it:

https://realpython.com/python-gil/

https://hackernoon.com/has-the-python-gil-been-slain-9440d28fa93d?gi=a9cc9f35638c 

So the problem reported could not be considered as an OpenVINO bug.


Regards,

Rizal


0 Kudos
Rizal_Intel
Moderator
1,821 Views

Hi Aankesh,


Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


0 Kudos
Aankesh
Employee
1,813 Views

hi Rizal,

This issue is happening in c++ as well. Please validate from your end also.

 

-Aankesh 

0 Kudos
Rizal_Intel
Moderator
1,788 Views

Hi Aankesh,


Would you kindly provide more details for the issue in C++?

Please provide some replication code if possible for me to verify this issue on C++.


Regards,

Rizal


0 Kudos
Rizal_Intel
Moderator
1,755 Views

Hi Aankesh,


Thank you for your question.

Intel will no longer monitor this thread . If you need any additional information from Intel, please submit a new question. 


Regards,

Rizal


0 Kudos
Reply