H.264 Decoder B-Frame Problem

[icomm]

Hi,

We're working on an RTP depacketizer for H.264. Everything works fine, except in streams, containing B-Frames (in this case Profile Main, Level 2.1).

The B-Frames are produced in following order, by the decoder: IBPBPB, which is fine the way we understand it, except that the B-Frames are displaced and somewhat distorted around the edges.
Playing this sequence, makes the video seem jittery and jumpy. If we take out the B-Frames the video is smooth again, but of course missing almost half the frames. That's apparently not acceptable.

What we have checked:
* No packet loss (we're using tcp for such tests)
* the stride / alignment is correct
* the timestamps of the VideoData::GetTime are ascending

We are using IPP samples 6.0.79, I've attached an image of the distorted B-Frame. As you can see PC-Guy isn't all that happy either.

If you need a capture of the stream, then we can provide that if necessary.

Thank you,

Bernd

[Vladimir_Dudnik]

Thanks Bernd,

yes, this is unexpected behaviour,H.264 stream would be very useful for analises.

Regards,
Vladimir

[icomm]

Hi Vladimir,

I attached the raw H.264 stream (I renamed it to .txt, because the server wouldn't let me upload other file extensions).

SPS/PPS is right in the beginning so it should play fine in the sample player.

Thanks,

Bernd

[Vladimir_Dudnik]

Hi Bernd,

our H.264 experts have looked at your stream, there is their comment:

It is incorrect stream. That is why decoder output is jittery and jumpy. I guess there is an error on RTP packetaizer or depacketaizer level. Frame order in this stream is incorrect. There it contains frames in display order IBPBPBP (I IDR frame with frame_num 0, B frame_num 2, P frame_num 1), but frames should be passed in decoder in decoding order - IPBPBPBPB. Because second display frame (with type B) uses I and P frames as references and P frame should be decoded before this B.

Regards,
Vladimir

[icomm]

Hi Vladimir,

Thanks so much for your time.
We looked around a bit and found a small little sidenote in the RFC that states:

"
Because H.264 allows the decoding order to be
different from the display order, values of RTP timestamps may
not be monotonically non-decreasing as a function of RTP
sequence numbers. Furthermore, the value for interarrival
jitter reported in the RTCP reports may not be a trustworthy
indication of the network performance, as the calculation rules
for interarrival jitter (section 6.4.1 of RFC 3550) assume that
the RTP timestamp of a packet is directly proportional to its
transmission time.
"

Just so people who try this won't make the same mistake we did!
The decoding order is dominated by the sequence number and not the timestamp, which can be non-ascending, but it is of course later used for display order.


Thank You,

Bernd