our H.264 experts have looked at your stream, there is their comment:
It is incorrect stream. That is why decoder output is jittery and jumpy. I guess there is an error on RTP packetaizer or depacketaizer level. Frame order in this stream is incorrect. There it contains frames in display order IBPBPBP (I IDR frame with frame_num 0, B frame_num 2, P frame_num 1), but frames should be passed in decoder in decoding order - IPBPBPBPB. Because second display frame (with type B) uses I and P frames as references and P frame should be decoded before this B.