H264 Decoder latency (again)

[Carsten_K_]

Hi,

i have seen many topics on H264 decoder latency in this forum, but my problem is not solved yet.

I feed the decoder with standard H264- IPPPP streams. I try to decode the stream on a picture by picture base (i allways stuff in a complete frame!), but the decoder does NOT return the decoded frame immedeatly. In many cases he expectes "more data".

My decoding parameters are:

• NO B-Frames!
• m_mfxBS.DataFlag = MFX_BITSTREAM_COMPLETE_FRAME;
• m_mfxVideoParams.AsyncDepth = 1;
•  NumThread = 1 what is not required!

First topic/question/bug:

When i put in the first IDR frame (SPS+PPS+SliceHdr+IDR Frame Data) i expect that DecodeFrameAsync(..)  returns with MFX_NO_ERROR and the decoded frame is available immediately. THERE IS NO REASON (!!!NO!!!!!) that  DecodeFrameAsync returns with MFX_ERR_MORE_DATA!

In an IPPPPP stream should every frame  be avaible immediately after DecodeFrameAsync()

When i decode streams, wich were encoded with the MediaSDK (with  mfxExtCodingOption::MaxDecFrameBuffering = 1), there is no problem, the decoder behaves "normally" and the decoded frame is returned immediately. BUT i have many streams (wich were not Media SDK encoded), where the DecodeFrameAsync returns with MFX_ERR_MORE_DATA. In this case i had to stuff in up to 20 Bytes of the following P-Frame (more than the complete slice header of the following P) to retrieve the decoded IDR. THIS IS NOT NESSARY, and in streaming situations with low fps rates (say 1 fps) unacceptable, because this behavior adds a 1s latency to the decoding path! I cant wait 1s for the next frame ... !!!

I add screenshots of  the SPS+PPS+SIdr-Slice HDR  of the two streams later. One is called Intel.h264 (wich returns MFX_NO_ERROR) the other is called error.h264 ( wich returns MFX_ERR_MORE_DATA). As you can see, the only difference between the streams (except some unimportant vui_flags), is the pic_order_cnt_type.

Second topic/question/bug:

In the stream error.h264 you can see, that the sequence_parameter_set says

• max_num_ref_frames =1
• vui_parameters_presesent=1
• -> bitstream_restriction_flag=1
• -> -> max_dec_frame_buffering=1

The call  

   // calculate number of surfaces required for decoder
    sts = m_pmfxDEC->QueryIOSurf(&m_mfxVideoParams, &Request);

returns with Request.NumFrameSuggested=10 ! Why???? The SPS said: max_num_ref_frames =1 and max_dec_frame_buffering=1, so NumFramesSuggested should be 2! This results in a big waiste of memory (in multi channel systems) and forces me to parse the headers myself again to retrieve the CORRECT NumFramesSuggested! :(

Do you have any ideas to fix the problems???

If your devellopers need teststreams to fix the BUGS, i can post some ...

Screenshots

Error.h264:

Intel.h264

[Carsten_K_]

There was something wrong with the pictures.. here they are:

Error.h264

Intel.h264

[OTorg]

Can't see your screenshots. Please, upload Intel.h264 and error.h264 (small portion with SPS, PPS and a few frames is quite enough).

[Petter_L_Intel]

Hi Carten,

please provide the streams so that we can explore the behavior locally. Please also share the graphics driver version and HW you are using.

Regarding your question about required # of decode surface buffers. This is a known limitation, due to underlying HW behavior (and graphics driver)  a greater number of surface buffers are allocated than is actually needed. There are plans to try to resolve this in future HW platforms and driver drops.

Regards,
Petter 

[Oh_i_]

Hi Carten,

I faced the same issues... about the first:

I just made a "fake" call to the decoder, if it replies "more data is needed", it takes no time and I can decode 1 frame at a time, with no latency! (i think it required a new "surface") it returns the decoded frame fully.

sbout the second, i guess it is a bug... usually it is not critical

[Oh_i_]

Hi Carten,

I faced the same issues... about the first:

I just made a "fake" call to the decoder, if it rreplies "more data is needed", it takes no time and I can decode 1 frame at a time, with no latency! (i think it required a new "surface", but ut returns the decoded frame fully.