Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Long latency for H.264Decoder

steven-lin
Beginner
762 Views

For video conference scenario, I have two GraphEdit as following:
Camera-->My_HW_H.264_Encoder-->Intel_Media_SDK_H.264_Decoder-->Enhanced_Video_Renderer
Camera-->My_HW_H.264_Encoder-->Software_H.264_Decoder-->Enhanced_Video_Renderer
SDK(Version 2.0.12.24071)

In the Intel_Media_SDK_H.264_Decoder scenario, the total latency is about 610ms.
In the Software_H.264_Decoder scenario, the latency is about 260ms.

I found that, in the first 11 frame, after DecodeFrameAsync(), the return value is MFX_ERR_MORE_DATA.
until the 12th frame, the return value is OK.
Is it the sequence header issue for first frame? If yes, can I set the preload parameters?
Havethe other reason on this? How to decrease the latency?

Thanks! --Steven

0 Kudos
4 Replies
Eric_S_Intel
Employee
762 Views

Hi Steven.

The Intel Media SDK is optimized for off line video transcoding and not video conferencing. To improve trancoding performance and avoid synchronization related issues, the decoder integrates quite a bit of buffering. This increases the latency. In addition, the stock sample filters that you are using do not have any mechanism for low latency operations. They are intended to be used for educational purposes only.

Internally, we are investigating what can be done to improve the latency for this usage model. Ill post a message when theres news on this front.

-Eric

0 Kudos
steven-lin
Beginner
762 Views

Hi Eric,

I'm going to using Intel_Media_SDK_H.264_Encoder for video conferencing.

Form my initial testing,

Camera-->Soft_MJPEG_Decoder-->Intel_Media_SDK_H.264_Encoder-->Soft_H.264_Decoder-->Enhanced_Video_Renderer

The result doesn't meet my expectation. Latencyis more than450ms.

Is the Intel_Media_SDK_H.264_Encoder suited for video conferencing?

Then I can adjust/modify the Encoder filter to decrease the latency.

If yes, could you give me some direction?

Thanks a lot!

Best Regards,

Steven

0 Kudos
Eric_S_Intel
Employee
762 Views

Hi Steven,

Is the reported 450ms attributed to just the encoder or the whole graph?

Like I said in the previous reply the Media SDK is more suited for off-line transcoding than it is for video conferencing.

We are actively exploring possibilities to improve the latency for this usage model. Ill post when we come to a definitive conclusion

-Eric

0 Kudos
steven-lin
Beginner
762 Views
Hi Eric,

Whole graph, for VC application,thereareextra latency, ie. networking.
I'll wait for your new post..Thanks a lot!


Steven
0 Kudos
Reply