Pictures references outside IDR (data stream wise) when using MFX_GOP_STRICT

[cedronius__anders]

Linux 20.04

MSDK -> 20.5.1

When setting the GopOptFlag MFX_GOP_STRICT there are frames delivered after each IDR referencing to times before the IDR.

In my print-out I divide the PTS and DTS to become a picture number.

As you see picture numbered 24 is a IDR as expected however the following B-Frames refer to times prior to the IDR making it difficult to create segment boundaries for creating for example HLS segments.

Mode: AVC(H.264)
The GOP settings I use:

GopPicSize = 24
GopRefDist = 3
IdrInterval = 0
GopOptFlag = MFX_GOP_CLOSED | MFX_GOP_STRICT

Got encoded image, pts=0 dts=0 (dts is negative but the printf is making it 0) IDR
Got encoded image, pts=3 dts=0 P-Frame
Got encoded image, pts=1 dts=1 B-Frame
Got encoded image, pts=2 dts=2 B-Frame
Got encoded image, pts=6 dts=3 P-Frame
Got encoded image, pts=4 dts=4 B-Frame
Got encoded image, pts=5 dts=5 B-Frame
Got encoded image, pts=9 dts=6 P-Frame
Got encoded image, pts=7 dts=7 B-Frame
Got encoded image, pts=8 dts=8 B-Frame
Got encoded image, pts=12 dts=9 P-Frame
Got encoded image, pts=10 dts=10 B-Frame
Got encoded image, pts=11 dts=11 B-Frame
Got encoded image, pts=15 dts=12 P-Frame
Got encoded image, pts=13 dts=13 B-Frame
Got encoded image, pts=14 dts=14 B-Frame
Got encoded image, pts=18 dts=15 P-Frame
Got encoded image, pts=16 dts=16 B-Frame
Got encoded image, pts=17 dts=17 B-Frame
Got encoded image, pts=21 dts=18 P-Frame
Got encoded image, pts=19 dts=19 B-Frame
Got encoded image, pts=20 dts=20 B-Frame
Got encoded image, pts=24 dts=21 IDR
Got encoded image, pts=22 dts=22 B-Frame
Got encoded image, pts=23 dts=23 B-Frame
Got encoded image, pts=27 dts=24 P-Frame

The same code without MFX_GOP_STRICT set produces a code-stream without any references to inside the GOP from outside the IDR NAL data.

GopOptFlag = MFX_GOP_CLOSED

Got encoded image, pts=0 dts=0 IDR
Got encoded image, pts=3 dts=0 P-Frame
Got encoded image, pts=1 dts=1 B-Frame
Got encoded image, pts=2 dts=2 B-Frame
Got encoded image, pts=6 dts=3 P-Frame
Got encoded image, pts=4 dts=4 B-Frame
Got encoded image, pts=5 dts=5 B-Frame
Got encoded image, pts=9 dts=6 P-Frame
Got encoded image, pts=7 dts=7 B-Frame
Got encoded image, pts=8 dts=8 B-Frame
Got encoded image, pts=12 dts=9 P-Frame
Got encoded image, pts=10 dts=10 B-Frame
Got encoded image, pts=11 dts=11 B-Frame
Got encoded image, pts=15 dts=12 P-Frame
Got encoded image, pts=13 dts=13 B-Frame
Got encoded image, pts=14 dts=14 B-Frame
Got encoded image, pts=18 dts=15 P-Frame
Got encoded image, pts=16 dts=16 B-Frame
Got encoded image, pts=17 dts=17 B-Frame
Got encoded image, pts=21 dts=18 P-Frame
Got encoded image, pts=19 dts=19 B-Frame
Got encoded image, pts=20 dts=20 B-Frame
Got encoded image, pts=23 dts=21 P-Frame
Got encoded image, pts=22 dts=22 B-Frame
Got encoded image, pts=24 dts=23 IDR
Got encoded image, pts=27 dts=24 P-Frame
Got encoded image, pts=25 dts=25 B-Frame
Got encoded image, pts=26 dts=26 B-Frame
Got encoded image, pts=30 dts=27 P-Frame

The issue has been discussed previously in this thread.

H-264-GOP-structure-question 

However many years has passed and no further information is provided. 

What is Intels view on this?  Working as it should?  Broken?

Thanks for helping.

/Anders.

[ArunJ_Intel]

Hi

We are forwarding this case to subject matter experts. They will get back to you soon regarding this.

Thanks

Arun

[Mark_L_Intel1]

Hi Anders,

Let me read the previous thread to understand more about this issue.

At the same time, do you happen to have any reproducer?

Mark

[cedronius__anders]

Hi Mark,

setting these parameters in the GOP control would reproduce the GOP pattern mentioned above

GopPicSize = 24
GopRefDist = 3
IdrInterval = 0
GopOptFlag = MFX_GOP_CLOSED | MFX_GOP_STRICT

 However if you are unable to set those parameters yourself I can put together some C++ project that does that. Just let me know.

/A 

[Mark_L_Intel1]

Thanks Anders,

It would be better if you can create a reproducer, I can do it but it will cost time and also if my reproducer has an error, it would cost more time to communicate.

If we do it with sample_encode, what would be the sample_encode command line and input video?

Mark

[cedronius__anders]

No problem,

Give me a day or two and I'll put something together.

/Anders.

[Mark_L_Intel1]

Thanks Anders,

If it takes too much time, let me know and we can work together.

Mark

[cedronius__anders]

OK made a project that illustrates the 'problem' in a simple way.

https://github.com/Unit-X/qsv_enc_lab

See the source file for details but as I write ->

//Reproducing the out of GOP backward referencing pictures add MFX_GOP_STRICT to the GopOptFlag

// Without MFX_GOP_STRICT the output.h264 produces ->

/*
FRAME type B   PTS: 84600 DTS 84600 Frame number: 49
FRAME type P   PTS: 88200 DTS 86400 Frame number: 50
FRAME type IDR PTS: 90000 DTS 88200 Frame number: 51
FRAME type P   PTS: 95400 DTS 90000 Frame number: 52
FRAME type B   PTS: 91800 DTS 91800 Frame number: 53
FRAME type B   PTS: 93600 DTS 93600 Frame number: 54
 */

// With MFX_GOP_STRICT set the output.h264 produces ->

/*
FRAME type B   PTS: 82800 DTS 82800 Frame number: 48
FRAME type B   PTS: 84600 DTS 84600 Frame number: 49
FRAME type IDR PTS: 90000 DTS 86400 Frame number: 50
FRAME type B   PTS: 88200 DTS 88200 Frame number: 51
FRAME type P   PTS: 95400 DTS 90000 Frame number: 52
 */

[Mark_L_Intel1]

Thanks so much and really appreciated.

Let me try it on my side and If confirmed, I will submitted a bug. Will keep you updated.

Mark Liu

[Mark_L_Intel1]

Hi Anders,

Yes, I tried your code and I can reproduce the behavior you described.

I will send this to dev team for investigation, for now, I am not sure if this is expected or a bug, I will keep you updated.

Just 2 more question:

• For the input yuv file, is it a normal video and just duplicate all the static image from the first frame?
• I checked the source file and didn't find anything special for encoding configuration, can you confirm?

Mark

[cedronius__anders]

Hi Mark,

Yes that is correct I just feed the same image to the encoder over and over.

And regarding the encoder configuration, it's taken from intels simple example encoder and the parameters are the same. The addition is the GOP structure parameters. 

Talk to you later and will be great to hear what they say. 

/Anders

[Pamela_H_Intel]

Apologies for the delays. Media SDK functionality has been superseded by oneVPL, FFmpeg and GStreamer.

For more information see the oneVPL Landing Page: https://www.intel.com/content/www/us/en/developer/tools/oneapi/onevpl.html where you will find, among other information:

oneVPL version 2.x is an API that is the successor to Intel® Media SDK. The Intel Media SDK project is no longer active and will not be updated. Packages and downloads are still available on GitHub*. For continued support and access to new features, Intel Media SDK users are encouraged to move to oneVPL as soon as possible.

·       Upgrade from Intel Media SDK to oneVPL

·       Intel Media SDK Discontinuation Notice

·       Download from GitHub

Let us know if you have any questions about oneVPL or how to convert from MediaSDK.