Cannot Decode H264 Stream from RTP with FFMPEG

Martin_R_2 · ‎03-22-2016

I try to integrate the Intel Media SDK in our software and want buy a license for it. But it didn't work in the test with the trial version of Media SDK 2016. I setup the Intel Media SDK. The samples work an the devices are accessable. Also the compilation with ffmpeg work. But if we change the codec to h264_qsv instead of h264 no picture will be shown, also no error comment on stderr was shown.

The first problem was the init. No passed packet makes an successfull init. The result ist every time the same: MFX_ERR_MORE_DATA. I changed the init of ffmpeg to manually set the format. Now the init was successful but, now MFXVideoDecode_DecodeFrameAsync always return's -> MFX_ERR_MORE_DATA.

I use the trial Media SDK 2016 with the preferred CentOS linux and the latest source of FFMpeg 2.8.

Please help!!!

david123456 · ‎03-22-2016

We also met this issue. The input stream is mpeg2 (file input is OK). Msdk is MSS2015 or 2016. OS is cent7.1. Processor is haswell.

Martin_R_2 · ‎03-23-2016

Dear David,
thanks for the answer, our OS Version is also CentOS 7.1 and the processor is haskwell, have you found a solution for the problem?
Best Regards
Martin

david123456 · ‎03-23-2016

Dear Martin,

We has no idea to solve the problem. We see from intel known issues that the stream input for QSV decoder may work abnormal.

Anna_B_Intel · ‎03-23-2016

Hi Martin and David,

It seems like an input stream format is incompatible for Media SDK. Media SDK works with start code prefixed (Annex B) input format. You can convert your input bitstream to Annex B format and try to use h264_qsv again.

Best wishes, Anna

Giri_b_ · ‎03-23-2016

what is Annex B format..? where it mentioned..?

Giri_b_ · ‎03-23-2016

ok..I understood,I think that "Annex B
Byte stream format " located @ h264 standard , is it correct...?

Martin_R_2 · ‎03-23-2016

Yes i used Annex B format.

Martin_R_2 · ‎03-23-2016

After i parsing the rtp stream an annex b stream was passed to the decoder, with normal h264 decoder of ffmpeg it works. The buffer we put to the deocder contains always the whole frame.

david123456 · ‎03-23-2016

Hi, Anna B.

Is there similar Annex B format for MPEG2 stream?

Anna_B_Intel · ‎03-23-2016

Hi Giri,

Yes, absolutely, Annex is a bitstream format. More information about it are available at the h.264 standard.

Martin,

Now I don't confident is it FFmpeg or Media SDK problem. I need some time for investigations. I'll come back with an answer a bit later.

David,

No, Mpeg2 has only one input bitstream format.

Martin_R_2 · ‎03-23-2016

I found the Problem. After parsing the RTP i had to add codec extra data (found in sdp definition) after nal type 5. If h264 stream was parsed by ffmpeg it was added, but the h264_qsv codec don't add it, because it use the intel specific parsing method. Adding the extra data transcode the stream to a real annex b format.

This link helps me to work out the solution:

http://ffmpeg.org/pipermail/ffmpeg-devel/2012-February/120421.html

Anna_B_Intel · ‎03-24-2016

Martin,

Great! Glad to know that you solved the problem.

Best wishes, Anna

Giri_b_ · ‎03-24-2016

Dear Anna B,

I followed the steps mentioned in the getting started guide for installing the Intel media server studio SDK.

1) Version used: MediaServerStudioEssentials2016.tar.gz

2) Here are details about system:

i) Model name: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (SKYLAKE PROCESSOR) &

x86_64 CPU : 64-bit

ii) Linux_Version used : CentOS Linux release 7.1.1503

3) checked few basic steps like :

i) [skylake@localhost user]$ lsmod | grep i915

Checked i915 DRM module is installed

ii) [squidskylake@localhost ffmpeg-3.0]$ lspci -nn -s 00:02.0
00:02.0 VGA compatible controller [0300]: Intel Corporation Sky Lake Integrated Graphics [8086:1912] (rev 06)

I tried to select/enter into MSS patched kernel while booting , now I am getting like

4) [root@localhost ffmpeg-3.0]$ uname -ar
Linux localhost.localdomain 3.10.0-229.1.2.47109.MSSr1.e17.centos.x86_64 #1 SMP Fri Mar 18 07:50:19 IST 2016 x86_64 x86_64 x86_64 GNU/Linux

5) [root@localhost ffmpeg-3.0]$ uname -r

3.10.0-229.1.2.47109.MSSr1.e17.centos.x86_64

6) configured libmfx library and checked hw accelerators :

[skylake@localhost ffmpeg3.0]$ ./ffmpeg -codecs | grep 'qsv'

Displaying all existing H264_qsv & HEVC_qsv encoders & decoders

7) but find some problem while running vainfo as follows.

[skylake@localhost ffmpeg-3.0]# vainfo

libva info: VA-API version 0.99.0
Xlib: extension "XFree86-DRI" missing on display ":0".
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
vaInitialize failed with error code -1 (unknown libva error),exit

can you please help me regarding above issues.

Thanks & Regards,

Giri.

Giri_b_ · ‎03-24-2016

Dear Anna B,

I am unable to use intel HW engines for AVC decoder with ffmpeg on skylake pc ,

Regards,

Giri.

Giri_b_ · ‎03-24-2016

Dear Martin,

Can you please list down the steps which you are followed to install Media SDK with Centos linux,

I am trying to use intel hw accelerators with latest source of FFMPEG 3.0 version.but unable to use intel hw accelerators,

I mentioned all the system specifications and issues in above mail thread given to Anna B..

Regards,

Giri

Giri_b_ · ‎03-25-2016

Dear all,

Right now i am getting errors like :

[user@localhost ffmpeg-3.0]$ ./ffmpeg -i /home/xyz/tractor.264 -vcodec h264_qsv /home/xyz/out1.yuv
ffmpeg version 3.0 Copyright (c) 2000-2016 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
configuration: --enable-libmfx --enable-nonfree
libavutil 55. 17.103 / 55. 17.103
libavcodec 57. 24.102 / 57. 24.102
libavformat 57. 25.100 / 57. 25.100
libavdevice 57. 0.101 / 57. 0.101
libavfilter 6. 31.100 / 6. 31.100
libswscale 4. 0.100 / 4. 0.100
libswresample 2. 0.101 / 2. 0.101
[h264 @ 0x23a6380] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from '/home/xyz/tractor.264':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: h264 (High), yuv420p, 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD128 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD129 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD130 fd open failed
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card0 vaInitialize failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card1 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card2 fd open failed
[h264_qsv @ 0x245d4c0] Selected ratecontrol mode is not supported by the QSV runtime. Choose a different mode.
Output #0, rawvideo, to '/home/xyz/out1.yuv':
Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, 25 fps
Metadata:
encoder : Lavc57.24.102 h264_qsv
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

regards,

Giri.

Anna_B_Intel · ‎03-25-2016

Hi Giri,

Media Server Studio 2016 doesn't support Skylake yet. Media SDK 2016 supports Skylake but it's for Windows only.

Best wishes, Anna

Giri_b_ · ‎03-25-2016

Dear Anna B,

ok,fine , thank you very much..

but even vainfo is also not working right..?

Regards,

Giri.

Martin_R_2 · ‎03-25-2016

Dear Giri,

i installed the media sdk on the steps in following manual

I compiled it with latest git version of ffmpeg 2.8 branch.
git clone -b release/2.8 https://github.com/FFmpeg/FFmpeg.git

Best Regards
Martin

Giri_b_ · ‎03-25-2016

Dear Martin,

Thank you very much,

Can you please send me details about your system(haswell/boradwell ?) and which OS r u using?

Regards,

Giri.