Video Conference setup on h264 decoder has huge latency

[minyi]

Hi,

I am trying to use MSDK HW h264 decoder for video conference usage on win32 platform. I tried to use MFXVideoDECODE::DecodeFrameAsync interface to do the job.

I followed https://software.intel.com/en-us/articles/video-conferencing-features-of-intel-media-software-development-kit  to config mfxVideoParam::AsyncDepth = 1 and set mfxBitStream::DataFlag = MFX_BITSTREAM_COMPLETE_FRAME each time I feed in a new frame.

But I continuously get MFX_ERR_MORE_DATA errors for the first 15~18 frames. I can only get decoded frames after I feed more frames. This cause huge delay to the system.

The encoded stream is not from MSDK, but from other h264 encoder. But it is encoded in IPPP mode and I can use other decoder(like MainConcept) to decode it frame by frame.

I attached the stream FYI.

Please anyone can help me to find out how to setup MSDK decoder to get frame-by-frame decoder working thus fulfill the video conference requirement.

Thanks,

Best Regards

Minyi

[Surbhi_M_Intel]

Hi Minyi, 

Not sure, which sample you are looking at for video conferencing but you can look at the existing video conferencing sample to do the job, present in the deprecated samples package here. To debug the issue. can you please let us know the platform details(CPU, graphics, driver version, OS), and Media SDK version. We will try to see what the issue is at our end as well with the stream you have provided. 

Thanks,
Surbhi
 

[minyi]

Hi Surbhi,

I tested on

CPU: i5-3320M

Graphic: Intel(R) HD Graphics 4000 with available Memory 1696MB(shared System Memory 1632MB)

Driver Version: 9.17.10.2843

OS: Win7 Professional with Service Pack 1

MSDK Versoin: 1.0

When I used both MSDK devices as encoder and decoder, there was no issue.

But when I used iOS encoded H264 stream as input, the decoder delay issue happened. When I switched to MainConcept H264 decoder, there was be no issue.

The major differences between MSDK encoded stream and other stream I can find is that

1. MSDK stream has max_dec_frame_buffering number set in SPS.

2. MSDK stream has slice type 7(I-Slice) and 5(P-Slice) while the other stream uses slice type 2(I-Slice) and 0(P-Slice)

I don't know if that causes the issue.

Thanks,

Minyi

[Surbhi_M_Intel]

Hi Minyi,

I see you have very old driver installed on your system. Can you please update it to latest.
Drivers for your system - https://downloadcenter.intel.com/search?keyword=3rd+Generation+Intel%c2%ae+Core%e2%84%a2+Processors+with+Intel%c2%ae+HD+Graphics+4000
Generally with every driver release lot of features are added and performance issues are fixed. This might be an issue in your case as well.
Which Media SDK/INDE version are you using?

I am discussing this issue with experts and will get back to you once we have a conclusion. It will be great if you can send us the input(YUV) as well so that we can make the comparison and understand why the decoder latency is high.

Thanks,
Surbhi

[Surbhi_M_Intel]

Hi Minyi,

Couple of workarounds-

• You can set the DecodedOrder member = 1 in mfxInfoMFX structure, decoder will return the frames with no delay. But it will be in the same order as frames are in bitstream. So there should be no B frames, if you do have reordering in the bitstream then application need to handle that.
• If you have a way to tune the encoder then set the DPB buffer size to be less, so that the delay reduces. Set max_dec_fram_buffering =1 in the SPS of the stream. This affects the DPB buffer and therefore decoding delay.

Also, please install latest Media SDK and drivers to test on.

Thanks,
Surbhi

[minyi]

Hi Surbhi,

I tried to update my graphic driver.

I can't use the driver in your link because Lenovo Thinkpad can only use its own driver.

I installed the latest one from their website: 9.18.10.3359 (11/15/2013)

For the yuv, I can't dump it but you can check the stream I attached to see the content.

I tried to set DecodedOrder to 1. This parameter didn't give me fix. I still got delay during decoding.

I will try to find a way to set DPB to less and hope that can help.

Please let me know if you have any other ways to shorten the delay.

Thanks,

Regards,

Minyi

[minyi]

Hi Surbhi,

I tried to add max_dec_fram_buffering parameter into the SPS before I feed it to MSDK decoder and the delay issue solved.

So I think the problem is this SPS parameter.

Because I can't setup this parameter in the 3rd party encoder and so far all parameters in MSDK decoder didn't work.

The only option I think will be change SPS stream header. What do you think?

Thanks,

Regards,

Minyi.

[Surbhi_M_Intel]

Hi Minyi, 

Sorry, it took me long to reply. You can ask Lenovo technical support for the latest drivers. We have seen customers requesting their vendors for the updated drivers in the past.
Do you have any other system 3rd or 4th generation hardware to test this feature by setting DecodeOrder =1?  If that couldn't be the case, then changing SPS header info is the only possible way we see it. 

Thanks,
Surbhi