H264 decoding problem

[alex78]

Deal all,

I have come across a very interesting problem with the H264 decoder and I hope anybody can put some light on it. I have a H264 decoder based on IPP 6.1.6 (both the IPP and the H264 decoder UMC sample code) working well for most H264 streams (the problem can be reproduced with 7.0.6 version as well). We have testeda live source (camera) producing a H264 stream and when we decode the stream, the colors seem to be somehow corrupted (please see the attached files). It does not seem to be the problem of the stream, because VLC for example decodes the stream with correct output.

I have uploaded an IDR frame (it starts with SPS and PPS data to correctly initialize the decoder):

http://software.intel.com/file/m/43503

The bitmap decoded by IPP (saved as a JPEG):

http://software.intel.com/file/m/43504

And a bitmap decoded by VLC, which is the expected result:

http://software.intel.com/file/m/43505

Does anybody know why the IPP result looks so strange?

Thanks very much in advance

Alex

[alex78]

Well, I thought that this problem will be interesting enough for anybody to look into it ;). What I would like to know (and what I cannot find out): Is there a bug in the H264 decoder causing this, or is there a unsupported H264 feature causing this, oris there a problem with the H264 data?

Well, as libavcodec (ffmpeg) decodes the frame with correct colors (see the images in my first post), something must be wrong. On the other hand, as the Intel IPP H264 decoder works well for most streams, there is definitely something weird or unusual in the H264 bitstream from that camera.

Thanks for _any_ suggestions.

Alex

[Yaroslav_Korchevsky]

Alex78,

I tried to decode your frame with the following decoders:
Elecard StreamEye - ( which personally I tend to trust )
CUDA H264 decoder
FFMPEG decoder 0.8.7

All of them decoded yor frame as IPP decoder does.

Which version of VLC do you use?

Does VLC send the same request to camera as your application does? The frame you attached is exactly the frame the VLC receives?

One more idea:
H264 specification is still under development. version of 2005 is not quite compatible with one 2010 in SPS header format.

Probably, your VLC build matches to newer H264 specification? than most of tools support.
Camera can send data in newer format as well.

Verify all of above. I beleive, you'll find the answer soon.

[alex78]

Jslav,

thanks very much for trying out other decoders. Your reply, indeed, pointed us to the right direction as VLC was receiving a slightly different stream! So this topic is now solved, thanks once more.

Youseem to have eperience with other H264 decoders, have you ever tried to do some performance comparisons? I am asking because when I try to decode a series of 1080p H264 frames, I can see that ffmpeg is up to 40-50 percent faster than an Intel IPP based decoder (I have put both decoders to single thread mode, I have no test results for multi-thread decoding).

Thanks

Alex

[Yaroslav_Korchevsky]

It depends of how do you use the decoder.
Each of decoders has the capability to use postprocessor (IPP) or frame allocator (FFMPEG)
to reduce decoded frame data copying.
In your case, possible, that IPP and FFMPEG were used with different postprocessing options.
Also, in FFMPEG there is an option to use or not videoframe border 16 pixels width from each side.
This option reduces machine CPU clocks required for decoding.

Commonly, I tend to agree with you, that FFMPEG has higher performance, but be careful.
Use it with best matching options.

PS: I have an experience with IPP 5.x dynamic build.
Now, I'm working on 7.x static build. So, it may change
Also, it may depend of particular CPU model.

As for multithreading, I have an experience to run multiple channels in multiple threads, not single channel
in multiple threads. So, I can't say what will happen if to use multithreaded instance of decoder.