Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

H.264 GOP structure question

Prashanth_D_
Beginner
1,162 Views

Hi,

We are seeing an unexpected behavior in the Intel Media SDK H.264 encoded bit-stream (hardware version) and looking for further clarification for the same. The details are as below.

We noticed that when GopOptFlag is set to MFX_GOP_STRICT, the GOP pattern generated by H.264 encoder is like this: IDRBBPBBPBBPBBP…

Here, the ‘B’ frames which are present immediately after the IDR frame, come before the IDR frame in the display order. Though, this pattern is technically correct, but we were expecting it to be in the pattern IDRPBBPBBPBBPBBP……where any frame after the IDR frame is displayed only after the IDR frame.

Could you please let us know how can we get the GOP pattern IDRPBBPBBPBBPBBP… when GopOptFlag is set to MFX_GOP_STRICT?

This behavior is reproducible in the sample application available in the SDK.

Additional details below:

Encoder Configuration

widthxheight: 1920x1080

bitrate: 4300 kbps

targetUsage: MFX_TARGETUSAGE_1

CodecLevel : MFX_LEVEL_AVC_4

CodecProfile: MFX_PROFILE_AVC_HIGH

GopPicSize : 60

GopOptFlag: MFX_GOP_STRICT

IdrInterval : 3

GopRefDist: 3

LookAheadDepth: 60

RateControlMethod: MFX_RATECONTROL_LA

I/O Pattern: MFX_IOPATTERN_IN_SYSTEM_MEMORY

 

SDK version        : MediaServerStudioEssentials2015R6

OS                       : Centos 7.1

Kernel version   : 3.10.0-229.1.2.39163.MSSr4.el7.centos.x86_64

Processor           : Intel(R) Core(TM) i7-4770R

 

Regards,

Prashanth Dixit

0 Kudos
7 Replies
Surbhi_M_Intel
Employee
1,162 Views

Hi Prashanth, 

I am able to replicate the behavior you have mentioned, however if you don't want the B frame to come before IDR frame(i.e. no reference to the next IDR frame) you can use closed GOP structure so that it doesn't use refer to IDR frame. 

I will check on how to generate  IDRPBBPBBPBBPBB GOP pattern, when GopOptFlag is set to MFX_GOP_STRICT and discuss it internally, will get back to you soon on this. 

Thanks,
Surbhi

0 Kudos
Prashanth_D_
Beginner
1,162 Views

Hi Surbhi,

Thanks for the reply.

Unfortunately we cannot set GopOptFlag to MFX_GOP_CLOSED. Actually the use case which we are trying to address is ABR segmentation which requires the presence of IDR frame at accurate frame positions i.e. every segment should end at the GOP boundary which is CLOSED. We believe this is guaranteed only when GopOptFlag is set to MFX_GOP_STRICT, i.e. in this case GOP will be CLOSED _and_ IDR frame occurs accurately at configured GopPicSize (GOP duration). Please let us know if our understanding is incorrect.

Waiting for your further findings.

Regards,

Prashanth Dixit

0 Kudos
Surbhi_M_Intel
Employee
1,162 Views

Hi Prahsanth, 

Not sure if you have read about GOP flag from the Media SDK manual, looks like what you want can be achieved by setting MFX_GOP_Closed. What I have understood is that you want the GOP to be closed and current frame shouldn't use frames from previous GOP structure at all. The presence of IDR frame after certain interval can be set by using IdrInterval and GOPPicsize.

Definition of MFX_GOP_CLOSED from Media SDK manual - "The encoder generates closed GOP if this flag is set. Frames in this GOP do not use frames in previous GOP as reference. The encoder generates open GOP if this flag is not set. In this GOP frames prior to the first frame of GOP in display order may use frames from previous GOP as reference. Frames subsequent to the first frame of GOP in display order do not use frames from previous GOP as reference. The AVC encoder ignores this flag if IdrInterval in mfxInfoMFX structure is set to 0, i.e. if every GOP starts from IDR frame. In this case, GOP is encoded as closed."

Can you set gop flag to be closed in your pipeline and see if the desired result can be achieved. Make sure you don't set Idr Interval to be 0. 
Please let us know if that solves the issue, we can take it from there. 

Thanks,
Surbhi

0 Kudos
Prashanth_D_
Beginner
1,162 Views

Hi Surbhi,

I think I have not articulated the problem clearly, sorry about that. Let me explain again with some more details using an example.

With occurrence of an IDR frame the previous GOP will be closed and the frames following IDR will not have any dependency on the frames of the previous GOP. The issue we are facing is slightly different and I have tried to explain it below.

For ABR use case, we want IDR frame exactly at the configured frame positions. For example, following can be one the encoder configuration which we can set.

Encoder Configuration:

GopPicSize : 60

IdrInterval : 3

GopRefDist: 3

Assuming 30 fps video, here GOP size is 2 seconds (60 frames) and IDR interval is 8 seconds (240 frames). With the above encoder configuration, we expect the IDR frames at 1st,  241st, 481st…. frame positions and I frames at 61st, 121st, 181s  ……. frame positions. For ABR use case, IDR frames must be present strictly at the configured positions. As per our understanding this behavior is guaranteed only when GopOptFlag is set to MFX_GOP_STRICT. According to the Media SDK manual (page 177), if we set the GopOptFlag to MFX_GOP_CLOSED, this behavior is not guaranteed. The encoder can adapt the GOP structure for better efficiency, whose range is constrained by parameter GopPicSize and GopRefDist etc. Which may eventually mean that 241st frame might not be an IDR, it might be off by few frames. Please let us know if our understanding here is incorrect.

So, we need to set the GopOptFlag to MFX_GOP_STRICT where the encoder must strictly follow the given GOP structure as defined by parameter GopPicSize, GopRefDist.

 

Although MFX_GOP_STRICT configuration guarantees the IDR frame interval, still there is a problem which was originally reported.

With MFX_GOP_STRICT configuration (along with IdrInterval configuration) we expect encoder to generate IDRPBBPBBPBBPBBP…… pattern. Here IDR frame come first in the display order and then rest of the frames.

However, encoder is generating IDRBBPBBPBBPBBP…pattern. Here, the two ‘B’ frames which are immediately following the IDR  frame come first in the display order, then IDR  frame and then rest of the frames. Although this is technically correct, it creates problem for ABR use case during switching across ABR profiles.

So, we need encoder to generate IDRPBBPBBPBBPBBP….. pattern even when GopOptFlag is set to MFX_GOP_STRICT.

Please let me know if I clarified the question right.

Regards,

Prashanth Dixit

0 Kudos
Surbhi_M_Intel
Employee
1,162 Views

Discussing this issue over email, will update the thread with a solution or workaround soon. 

Thanks,
Surbhi

0 Kudos
Gilles_V_
Beginner
1,162 Views

Hi all,

could you please update this post if a solution was found?

I am experiencing the same issue trying to do qsv encoding with HLS...

Thanks!

Gilles

0 Kudos
Gilles_V_
Beginner
1,162 Views

By the way, even when using MFX_GOP_CLOSED, I have the B frames coming before the I-Frames in Display order.

Is it wanted ?

0 Kudos
Reply