Cannot Decode H264 Stream from RTP with FFMPEG

[Martin_R_2]

I try to integrate the Intel Media SDK in our software and want buy a license for it. But it didn't work in the test with the trial version of Media SDK 2016. I setup the Intel Media SDK. The samples work an the devices are  accessable. Also the compilation with ffmpeg work. But if we change the codec to h264_qsv instead of h264 no picture will be shown, also no error comment on stderr was shown. 

The first problem was the init. No passed packet makes an successfull init. The result ist every time the same: MFX_ERR_MORE_DATA. I changed the init of ffmpeg to manually set the format. Now the init was successful but, now MFXVideoDecode_DecodeFrameAsync always return's -> MFX_ERR_MORE_DATA. 

I use the trial Media SDK 2016 with the preferred CentOS linux and the latest source of FFMpeg 2.8.

Please help!!!

[david123456]

We also met this issue. The input stream is mpeg2 (file input is OK). Msdk is MSS2015 or 2016. OS is cent7.1. Processor is haswell.
 

[Martin_R_2]

Dear David,
thanks for the answer, our OS Version is also CentOS 7.1 and the processor is haskwell, have you found a solution for the problem?
Best Regards
Martin

[david123456]

Dear Martin,

We has no idea to solve the problem. We see from intel known issues that the stream input for QSV decoder may work abnormal.

[Anna_B_Intel]

Hi Martin and David,

It seems like an input stream format is incompatible for Media SDK. Media SDK works with start code prefixed (Annex B) input format. You can convert your input bitstream to Annex B format and try to use h264_qsv again.

Best wishes, Anna    

[Giri_b_]

what is Annex B format..? where it mentioned..?

[Giri_b_]

ok..I understood,I think that "Annex B
Byte stream format " located @ h264 standard , is it correct...?

[Martin_R_2]

Yes i used Annex B format. 

[Martin_R_2]

After i parsing the rtp stream an annex b stream was passed to the decoder, with normal h264 decoder of ffmpeg it works. The buffer we put to the deocder contains always the whole frame. 

[david123456]

Hi, Anna B.

Is there similar Annex B format for MPEG2 stream?

[Anna_B_Intel]

Hi Giri,

Yes, absolutely, Annex is a bitstream format. More information about it are available at the h.264 standard.

Martin, 

Now I don't confident is it FFmpeg  or Media SDK problem. I need some time for investigations. I'll come back with an answer a bit later.

David,

No, Mpeg2 has only one input bitstream format.

[Martin_R_2]

I found the Problem. After parsing the RTP i had to add codec extra data (found in sdp definition) after nal type 5. If h264 stream was parsed by ffmpeg it was added, but  the h264_qsv codec don't add it, because it use the intel specific parsing method. Adding the extra data transcode the stream to a real annex b format.

This link helps me to work out the solution:

http://ffmpeg.org/pipermail/ffmpeg-devel/2012-February/120421.html

[Anna_B_Intel]

Martin,

Great! Glad to know that you solved the problem.

Best wishes, Anna 

[Giri_b_]

Dear Anna B,

I followed the steps mentioned in the getting started guide for installing the Intel media server studio SDK.

1) Version used: MediaServerStudioEssentials2016.tar.gz

2) Here are details about system:

      i) Model name: Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz (SKYLAKE PROCESSOR) & 

          x86_64 CPU : 64-bit 

     ii) Linux_Version used : CentOS Linux release 7.1.1503

3) checked few basic steps like :

     i)  [skylake@localhost user]$ lsmod | grep i915

          Checked i915 DRM module is installed

     ii) [squidskylake@localhost ffmpeg-3.0]$ lspci -nn -s 00:02.0
           00:02.0 VGA compatible controller [0300]: Intel Corporation Sky Lake      Integrated Graphics [8086:1912] (rev 06)

I tried to select/enter into MSS patched kernel while booting , now I am getting like

4) [root@localhost ffmpeg-3.0]$ uname -ar
    Linux localhost.localdomain  3.10.0-229.1.2.47109.MSSr1.e17.centos.x86_64 #1 SMP Fri Mar 18   07:50:19 IST 2016 x86_64 x86_64 x86_64 GNU/Linux

5) [root@localhost ffmpeg-3.0]$ uname -r

    3.10.0-229.1.2.47109.MSSr1.e17.centos.x86_64

6) configured libmfx library and checked hw accelerators :

       [skylake@localhost ffmpeg3.0]$ ./ffmpeg -codecs | grep 'qsv'

        Displaying all existing H264_qsv & HEVC_qsv encoders & decoders

7) but find some problem while running vainfo as follows.

[skylake@localhost ffmpeg-3.0]# vainfo

libva info: VA-API version 0.99.0
Xlib:  extension "XFree86-DRI" missing on display ":0".
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
vaInitialize failed with error code -1 (unknown libva error),exit

can you please help me regarding above issues.

Thanks & Regards,

Giri.

[Giri_b_]

Dear Anna B,

I am unable to use intel HW engines for AVC decoder with ffmpeg on skylake pc ,

Regards,

Giri.

[Giri_b_]

Dear Martin,

Can you please list down the steps which you are followed to install Media SDK with Centos linux,

I am trying to use intel hw accelerators with latest source of FFMPEG 3.0 version.but unable to use intel hw accelerators,

I mentioned all the system specifications and issues in above mail thread given to Anna B..

Regards,

Giri

[Giri_b_]

Dear all,

Right now i am getting errors like :

[user@localhost ffmpeg-3.0]$ ./ffmpeg -i /home/xyz/tractor.264 -vcodec h264_qsv /home/xyz/out1.yuv
ffmpeg version 3.0 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-4)
  configuration: --enable-libmfx --enable-nonfree
  libavutil      55. 17.103 / 55. 17.103
  libavcodec     57. 24.102 / 57. 24.102
  libavformat    57. 25.100 / 57. 25.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 31.100 /  6. 31.100
  libswscale      4.  0.100 /  4.  0.100
  libswresample   2.  0.101 /  2.  0.101
[h264 @ 0x23a6380] Stream #0: not enough frames to estimate rate; consider increasing probesize
Input #0, h264, from '/home/xyz/tractor.264':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: h264 (High), yuv420p, 1280x720, 25 fps, 25 tbr, 1200k tbn, 50 tbc
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD128 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD129 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/renderD130 fd open failed
libva info: VA-API version 0.99.0
libva info: va_getDriverName() returns -1
libva error: va_getDriverName() failed with unknown libva error,driver_name=(null)
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card0 vaInitialize failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card1 fd open failed
[h264_qsv @ 0x245d4c0] mfx init: /dev/dri/card2 fd open failed
[h264_qsv @ 0x245d4c0] Selected ratecontrol mode is not supported by the QSV runtime. Choose a different mode.
Output #0, rawvideo, to '/home/xyz/out1.yuv':
    Stream #0:0: Video: h264, none, q=2-31, 128 kb/s, 25 fps
    Metadata:
      encoder         : Lavc57.24.102 h264_qsv
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (h264_qsv))
Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height

regards,

Giri.

[Anna_B_Intel]

Hi Giri,

Media Server Studio 2016 doesn't support Skylake yet. Media SDK 2016 supports Skylake but it's for Windows only.

Best wishes, Anna

[Giri_b_]

Dear Anna B,

ok,fine , thank you very much..

but even  vainfo is also not working right..?   

Regards,

Giri.

[Martin_R_2]

Dear Giri,

i installed the media sdk on the steps in following manual

• http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/quicksync-video-ffmpeg-install-valid.pdf
• https://software.intel.com/sites/default/files/media_server_studio_getting_started_guide.pdf

I compiled it with latest git version of ffmpeg 2.8 branch.
git clone -b release/2.8 https://github.com/FFmpeg/FFmpeg.git

Best Regards
Martin

[Giri_b_]

Dear Martin,

Thank you very much,

Can you please send me details about your system(haswell/boradwell ?) and which OS r u using?

Regards,

Giri.