- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
When loading GPU as a device to run the inference, the app main thread is unresponsive for sometime. But this is not the case with CPU or other devices. The same behviour happens when loading complex models like semantic segmentation and object detection.
-Aankesh
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
This is a one-time overhead (order of few seconds) of compiling the OpenCL kernels.
The compilation happens upon loading the network to the GPU plugin and does not affect the inference time.
You can read more about GPU optimization here.
If you simply don’t want your main thread from freezing you could run the model asynchronously by running it on another thread.
Regards,
Rizal.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi,
the actual issue is, while the compilation or Running in Async mode, the main threads are getting chocked, even when we are spinning other thread for loading the model. This is seen especially when loading complex models and also durin the GPU device loading.
-Aankesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
The all thread freezing behavior is not expected.
However, you can use cl_cache method as a workaround to shorten the time for GPU loading.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Rizal,
thanks . We wanted to understand why the thread is getting chocked. Wanted to understand if any interrupts are being masked which we are not able to get it through our debugging.
-Aankesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
I did some testing, at it seems at least the keyboards interrupt is registered but deferred (executed later after the GPU plugin finished loading).
How was your debugging result, did the interrupts missing completely (not registered at all) or is it also deferred?
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi Rizal,
from our end, all the interrupts are getting blocked. We built a Python GUI app that connects to the openvino. The moment GPU load begins, the entire GUI is hung and all the signals are getting blocked until the gpu load is successful even though we spin a separate thread for the openvino process.
-Aankesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi Rizal,
do you have any updates for me on the issue?
-Aankesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Is there a way to 'cache' the compiled kernel? It is one of the deal breaker for demo as it hang way too long before anything works.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Raymond,
You can use cl_cache method as a workaround to shorten the time for subsequent GPU loading.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
We are still investigating this issue, we will inform you on any updates or development on the matter.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
The investigation shows that this script hits the well-known Python GIL problem, which is described in links below along with some clues on how to workaround it:
https://realpython.com/python-gil/
https://hackernoon.com/has-the-python-gil-been-slain-9440d28fa93d?gi=a9cc9f35638c
So the problem reported could not be considered as an OpenVINO bug.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
hi Rizal,
This issue is happening in c++ as well. Please validate from your end also.
-Aankesh
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
Would you kindly provide more details for the issue in C++?
Please provide some replication code if possible for me to verify this issue on C++.
Regards,
Rizal
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Aankesh,
Thank you for your question.
Intel will no longer monitor this thread . If you need any additional information from Intel, please submit a new question.
Regards,
Rizal

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page