I recalled there is some risk if modify code like i did, for example, when there is B frame in stream. Could you attached a piece of crashed stream?
Have youtried the suggestion of Jay, how it works?
Or other ideas, for example, reduce the delay by define smallDecFrameBuffer instead of 16, but not avoid the delay completely.
I'm very sorry to tell that no, IPP UMC team haven't plan to avoid the delay in the future official release.
I quotesome comment from our expert:
There is internal buffer which called DPB (Decoded Picture Buffer). By standard, decoder couldn't return frame until DPB is full enough (It'sneeded for correct reordering frames process). So, when there are empty slots in DPB GetFrame function returns
UMC_NOT_ENOUGH_DATA. When DPB is full enough GetFrame return UMC_OK and you can use returned frame.
For using H.264 decoder in video conference applicationsit's recommended to manually adjust DPB buffer size.
Size of DPB depends from level_idc (IPP H.264 encoder parameter) and picture's size.
For example, picture size is 320x280. When you set level_idc to 30 then DPB size will be ~12 frames. In this case, decoder could return first frame to application only after it's received and decode 12 frames. As result, long delay occurs. When you set level_idc to 10, then DPB size will be ~1-2 frames and delay will be short.
Another counter-example to the DPB needing to be full (ie. 12 frames) is that whenever an IDR frame is encountered, the DPB should be emptied and all previous reference frames should stop being used.
UMC H.264 decoder always stays at 12 and introduces artificially high latency just so it never has to return UMC_NOT_ENOUGH_DATA, which violates IDR frame behavior. If you decode an IDR frame, you're supposed to get it right away, not 11 frames in the past. This results in people finding seeking hard to implement with UMC and having to either hack it or resort to calling Reset() to forcefully clear the DPB, crashing in the process.
A full DPB makes internal multi-threading much easier to implement so I see the benefit, but if there are no plans to change this, could there at least be an optional setting to force low-latency behavior in the future?
Thank a lot for the suggesiton. I have been escalted the feature to the UMC developer team. Hope they may consider it some days.
Digged out one morepossibleworkaroundabout reduce the delay in decoder:
There is VideoDecoderParams::dPlaybackRate parameter (Ipp64f). By default dPlaybackRate equals 1.0. If dPlaybackRate doesn't equal 1.0 (at H.264 decoder Init) then decoder outputs in decoded order instead of display order. It allows to significantly reduce delay.