Media (Intel® oneAPI Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools from Intel. This includes Intel® oneAPI Video Processing Library and Intel® Media SDK.
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.

Ubuntu hangs - GPU stuck at 100% with MSDK application

tomasoh-edgeware
802 Views

Hi!

We are having some problem with a live encoding application based on MSDK, targeting Ubuntu 20.04.

The application uses a VPP block to convert (and possibly scale) incoming raw video frames from UYVY to NV12 and then encodes them to AVC. We usually encode 2 to 6 incoming video streams in parallel.

The application works without issues on some of our machines, but on one particular machine we sometimes experience a complete operating system freeze. Using the intel_gpu_top we can see that the GPU usage gets stuck on 100% some seconds before we loose contact with the machine (SSH stops working, it does not answer to ping and we have to reboot it by physically pressing the button). Most often the machine hangs when we have executed the application one or more times, then stopped it and executed it again. But it only happen once in a while and the application can sometimes run over a weekend on this machine without any problems.
We have tried to find clues in the system logs after having to physically reboot the OS, but without any luck.

Here is the system specification of the machine we are having trouble with:

$ uname -r
5.8.0-49-generic
$ cat /proc/cpuinfo | grep -i intel
...
model name : Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
...
$ lspci -nn | grep -i vga
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics [8086:9b41] (rev 02)
$ vainfo
libva info: VA-API version 1.11.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.11 (libva 2.11.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.3 (f0b7785)
...

 

 

Here are the specifications for two machines that we have not seen this behavior on:

$ uname -r
5.4.0-66-generic
$ cat /proc/cpuinfo | grep -i intel
...
model name : Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz
...
$ lspci -nn | grep -i vga
00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:1912] (rev 06)
$ vainfo
libva info: VA-API version 1.10.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_10
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.10 (libva 2.10.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 20.4.5 (a0bf03c)
...

 

and

 

$ uname -r
5.8.0-50-generic
$ cat /proc/cpuinfo | grep -i intel
...
model name : Intel(R) Core(TM) i5-10210U CPU @ 1.60GHz
...
$ lspci -nn | grep -i vga
00:02.0 VGA compatible controller [0300]: Intel Corporation UHD Graphics [8086:9b41] (rev 02)
$ vainfo
libva info: VA-API version 1.11.0
libva info: User environment variable requested driver 'iHD'
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_11
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.11 (libva 2.11.0)
vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 21.1.3 (461cea3)
...

(Yes, this one uses pretty much the same HW and SW as the machine we have trouble with, but we haven't seen any hangs on this one)

 

All machines run Ubuntu 20.04. Compiler version is gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0.

Do you have any idea what could cause this issue?
Any ideas on how to debug this?

Labels (1)
0 Kudos
7 Replies
RahulU_Intel
Moderator
777 Views

Hi, 


Thanks for posting in Intel Forums. Thanks for posting your query in quite detail. Can you try updating your Media SDK to the latest version and also send us the Sample Reproducer so that we can try reproducing at our end.


Thanks


tomasoh-edgeware
770 Views

Thanks for the reply,

 

The machine where we have the problem is already at the latest MSDK release (v21.1.3), as well as the other machine with identical HW specs. What we could test is to update the other machine with the i7-6700K to the latest MSDK and see if that will reproduce the problem on that machine.

 

What do you mean by "Sample Reproducer"? The actual application is part of a larger, commercial system, so I would prefer not to share it.

RahulU_Intel
Moderator
734 Views

Hi,


  We looked into your issue. Can you try/confirm below mentioned steps/queries,


  1. Can you try changing Kernel version to 5.8.0-50-generic/5.4.0-66-generic with iHD driver 21.1.3 and see if the issue persists.

  2. Could you try running the application as single stream and see if you are observing the same issue

  3. Are you facing any hang without color conversion? Also please try changing the pipeline and see if the issue still exists with single component. Say if the pipeline is, stream input-->vpp-->encode.  You can try removing the color conversion vpp component and run the pipeline and also you can try removing the encode part and again run the pipeline by keeping only one component at a time.


tomasoh-edgeware
719 Views

Thanks for the answer.

 

We will test running the application with the suggestions you have see if we can get some new insights.

RahulU_Intel
Moderator
694 Views

Hi,

We haven't heard back from you. Could you please confirm if the issue is resolved.


Thanks


tomasoh-edgeware
667 Views

Hi,

 

Unfortunately due to internal deadlines and such, we have not been able to do much testing or updating since last time. But the issue seem to persist. We will keep investigating and see if we can manage to find some more data on what is happening right before the hangs. Unfortunately this is a time consuming task, as it may take several hours or even days between two hangs. I will let you know if we find anything.

Thanks,

//Tomas

RahulU_Intel
Moderator
607 Views

Hi,


I have not heard back from you, so I will close this inquiry now. If you need further assistance, please post a new question.


Thanks and Regards


Reply