We are seeing an unexpected behavior in the Intel Media SDK H.264 encoded bit-stream (hardware version) and looking for further clarification for the same. The details are as below.
We noticed that when GopOptFlag is set to MFX_GOP_STRICT, the GOP pattern generated by H.264 encoder is like this: IDRBBPBBPBBPBBP…
Here, the ‘B’ frames which are present immediately after the IDR frame, come before the IDR frame in the display order. Though, this pattern is technically correct, but we were expecting it to be in the pattern IDRPBBPBBPBBPBBP……where any frame after the IDR frame is displayed only after the IDR frame.
Could you please let us know how can we get the GOP pattern IDRPBBPBBPBBPBBP… when GopOptFlag is set to MFX_GOP_STRICT?
This behavior is reproducible in the sample application available in the SDK.
Additional details below:
bitrate: 4300 kbps
CodecLevel : MFX_LEVEL_AVC_4
GopPicSize : 60
IdrInterval : 3
I/O Pattern: MFX_IOPATTERN_IN_SYSTEM_MEMORY
SDK version : MediaServerStudioEssentials2015R6
OS : Centos 7.1
Kernel version : 3.10.0-126.96.36.199163.MSSr4.el7.centos.x86_64
Processor : Intel(R) Core(TM) i7-4770R
I am able to replicate the behavior you have mentioned, however if you don't want the B frame to come before IDR frame(i.e. no reference to the next IDR frame) you can use closed GOP structure so that it doesn't use refer to IDR frame.
I will check on how to generate IDRPBBPBBPBBPBB GOP pattern, when GopOptFlag is set to MFX_GOP_STRICT and discuss it internally, will get back to you soon on this.
Thanks for the reply.
Unfortunately we cannot set GopOptFlag to MFX_GOP_CLOSED. Actually the use case which we are trying to address is ABR segmentation which requires the presence of IDR frame at accurate frame positions i.e. every segment should end at the GOP boundary which is CLOSED. We believe this is guaranteed only when GopOptFlag is set to MFX_GOP_STRICT, i.e. in this case GOP will be CLOSED _and_ IDR frame occurs accurately at configured GopPicSize (GOP duration). Please let us know if our understanding is incorrect.
Waiting for your further findings.
Not sure if you have read about GOP flag from the Media SDK manual, looks like what you want can be achieved by setting MFX_GOP_Closed. What I have understood is that you want the GOP to be closed and current frame shouldn't use frames from previous GOP structure at all. The presence of IDR frame after certain interval can be set by using IdrInterval and GOPPicsize.
Definition of MFX_GOP_CLOSED from Media SDK manual - "The encoder generates closed GOP if this flag is set. Frames in this GOP do not use frames in previous GOP as reference. The encoder generates open GOP if this flag is not set. In this GOP frames prior to the first frame of GOP in display order may use frames from previous GOP as reference. Frames subsequent to the first frame of GOP in display order do not use frames from previous GOP as reference. The AVC encoder ignores this flag if IdrInterval in mfxInfoMFX structure is set to 0, i.e. if every GOP starts from IDR frame. In this case, GOP is encoded as closed."
Can you set gop flag to be closed in your pipeline and see if the desired result can be achieved. Make sure you don't set Idr Interval to be 0.
Please let us know if that solves the issue, we can take it from there.
I think I have not articulated the problem clearly, sorry about that. Let me explain again with some more details using an example.
With occurrence of an IDR frame the previous GOP will be closed and the frames following IDR will not have any dependency on the frames of the previous GOP. The issue we are facing is slightly different and I have tried to explain it below.
For ABR use case, we want IDR frame exactly at the configured frame positions. For example, following can be one the encoder configuration which we can set.
GopPicSize : 60
IdrInterval : 3
Assuming 30 fps video, here GOP size is 2 seconds (60 frames) and IDR interval is 8 seconds (240 frames). With the above encoder configuration, we expect the IDR frames at 1st, 241st, 481st…. frame positions and I frames at 61st, 121st, 181s ……. frame positions. For ABR use case, IDR frames must be present strictly at the configured positions. As per our understanding this behavior is guaranteed only when GopOptFlag is set to MFX_GOP_STRICT. According to the Media SDK manual (page 177), if we set the GopOptFlag to MFX_GOP_CLOSED, this behavior is not guaranteed. The encoder can adapt the GOP structure for better efficiency, whose range is constrained by parameter GopPicSize and GopRefDist etc. Which may eventually mean that 241st frame might not be an IDR, it might be off by few frames. Please let us know if our understanding here is incorrect.
So, we need to set the GopOptFlag to MFX_GOP_STRICT where the encoder must strictly follow the given GOP structure as defined by parameter GopPicSize, GopRefDist.
Although MFX_GOP_STRICT configuration guarantees the IDR frame interval, still there is a problem which was originally reported.
With MFX_GOP_STRICT configuration (along with IdrInterval configuration) we expect encoder to generate IDRPBBPBBPBBPBBP…… pattern. Here IDR frame come first in the display order and then rest of the frames.
However, encoder is generating IDRBBPBBPBBPBBP…pattern. Here, the two ‘B’ frames which are immediately following the IDR frame come first in the display order, then IDR frame and then rest of the frames. Although this is technically correct, it creates problem for ABR use case during switching across ABR profiles.
So, we need encoder to generate IDRPBBPBBPBBPBBP….. pattern even when GopOptFlag is set to MFX_GOP_STRICT.
Please let me know if I clarified the question right.