Long latency for H.264Decoder

[steven-lin]

For video conference scenario, I have two GraphEdit as following:
Camera-->My_HW_H.264_Encoder-->Intel_Media_SDK_H.264_Decoder-->Enhanced_Video_Renderer
Camera-->My_HW_H.264_Encoder-->Software_H.264_Decoder-->Enhanced_Video_Renderer
SDK(Version 2.0.12.24071)

In the Intel_Media_SDK_H.264_Decoder scenario, the total latency is about 610ms.
In the Software_H.264_Decoder scenario, the latency is about 260ms.

I found that, in the first 11 frame, after DecodeFrameAsync(), the return value is MFX_ERR_MORE_DATA.
until the 12th frame, the return value is OK.
Is it the sequence header issue for first frame? If yes, can I set the preload parameters?
Havethe other reason on this? How to decrease the latency?

Thanks! --Steven

[Eric_S_Intel]

Hi Steven.

The Intel Media SDK is optimized for off line video transcoding and not video conferencing. To improve trancoding performance and avoid synchronization related issues, the decoder integrates quite a bit of buffering. This increases the latency. In addition, the stock sample filters that you are using do not have any mechanism for low latency operations. They are intended to be used for educational purposes only.

Internally, we are investigating what can be done to improve the latency for this usage model. Ill post a message when theres news on this front.

-Eric

[steven-lin]

Hi Eric,

I'm going to using Intel_Media_SDK_H.264_Encoder for video conferencing.

Form my initial testing,

Camera-->Soft_MJPEG_Decoder-->Intel_Media_SDK_H.264_Encoder-->Soft_H.264_Decoder-->Enhanced_Video_Renderer

The result doesn't meet my expectation. Latencyis more than450ms.

Is the Intel_Media_SDK_H.264_Encoder suited for video conferencing?

Then I can adjust/modify the Encoder filter to decrease the latency.

If yes, could you give me some direction?

Thanks a lot!

Best Regards,

Steven

[Eric_S_Intel]

Hi Steven,

Is the reported 450ms attributed to just the encoder or the whole graph?

Like I said in the previous reply the Media SDK is more suited for off-line transcoding than it is for video conferencing.

We are actively exploring possibilities to improve the latency for this usage model. Ill post when we come to a definitive conclusion

-Eric

[steven-lin]

Hi Eric,

Whole graph, for VC application,thereareextra latency, ie. networking.
I'll wait for your new post..Thanks a lot!


Steven