Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3072 Discussions

Pictures references outside IDR (data stream wise) when using MFX_GOP_STRICT

cedronius__anders
New Contributor I
2,272 Views

Linux 20.04

MSDK -> 20.5.1

When setting the GopOptFlag MFX_GOP_STRICT there are frames delivered after each IDR referencing to times before the IDR.

In my print-out I divide the PTS and DTS to become a picture number.

As you see picture numbered 24 is a IDR as expected however the following B-Frames refer to times prior to the IDR making it difficult to create segment boundaries for creating for example HLS segments.

Mode: AVC(H.264)
The GOP settings I use:

GopPicSize = 24
GopRefDist = 3
IdrInterval = 0
GopOptFlag = MFX_GOP_CLOSED | MFX_GOP_STRICT


Got encoded image, pts=0 dts=0 (dts is negative but the printf is making it 0) IDR
Got encoded image, pts=3 dts=0 P-Frame
Got encoded image, pts=1 dts=1 B-Frame
Got encoded image, pts=2 dts=2 B-Frame
Got encoded image, pts=6 dts=3 P-Frame
Got encoded image, pts=4 dts=4 B-Frame
Got encoded image, pts=5 dts=5 B-Frame
Got encoded image, pts=9 dts=6 P-Frame
Got encoded image, pts=7 dts=7 B-Frame
Got encoded image, pts=8 dts=8 B-Frame
Got encoded image, pts=12 dts=9 P-Frame
Got encoded image, pts=10 dts=10 B-Frame
Got encoded image, pts=11 dts=11 B-Frame
Got encoded image, pts=15 dts=12 P-Frame
Got encoded image, pts=13 dts=13 B-Frame
Got encoded image, pts=14 dts=14 B-Frame
Got encoded image, pts=18 dts=15 P-Frame
Got encoded image, pts=16 dts=16 B-Frame
Got encoded image, pts=17 dts=17 B-Frame
Got encoded image, pts=21 dts=18 P-Frame
Got encoded image, pts=19 dts=19 B-Frame
Got encoded image, pts=20 dts=20 B-Frame
Got encoded image, pts=24 dts=21 IDR
Got encoded image, pts=22 dts=22 B-Frame
Got encoded image, pts=23 dts=23 B-Frame
Got encoded image, pts=27 dts=24 P-Frame

The same code without MFX_GOP_STRICT set produces a code-stream without any references to inside the GOP from outside the IDR NAL data.

GopOptFlag = MFX_GOP_CLOSED

Got encoded image, pts=0 dts=0 IDR
Got encoded image, pts=3 dts=0 P-Frame
Got encoded image, pts=1 dts=1 B-Frame
Got encoded image, pts=2 dts=2 B-Frame
Got encoded image, pts=6 dts=3 P-Frame
Got encoded image, pts=4 dts=4 B-Frame
Got encoded image, pts=5 dts=5 B-Frame
Got encoded image, pts=9 dts=6 P-Frame
Got encoded image, pts=7 dts=7 B-Frame
Got encoded image, pts=8 dts=8 B-Frame
Got encoded image, pts=12 dts=9 P-Frame
Got encoded image, pts=10 dts=10 B-Frame
Got encoded image, pts=11 dts=11 B-Frame
Got encoded image, pts=15 dts=12 P-Frame
Got encoded image, pts=13 dts=13 B-Frame
Got encoded image, pts=14 dts=14 B-Frame
Got encoded image, pts=18 dts=15 P-Frame
Got encoded image, pts=16 dts=16 B-Frame
Got encoded image, pts=17 dts=17 B-Frame
Got encoded image, pts=21 dts=18 P-Frame
Got encoded image, pts=19 dts=19 B-Frame
Got encoded image, pts=20 dts=20 B-Frame
Got encoded image, pts=23 dts=21 P-Frame
Got encoded image, pts=22 dts=22 B-Frame
Got encoded image, pts=24 dts=23 IDR
Got encoded image, pts=27 dts=24 P-Frame
Got encoded image, pts=25 dts=25 B-Frame
Got encoded image, pts=26 dts=26 B-Frame
Got encoded image, pts=30 dts=27 P-Frame


The issue has been discussed previously in this thread.

H-264-GOP-structure-question 

However many years has passed and no further information is provided. 

What is Intels view on this?  Working as it should?  Broken?

Thanks for helping.

/Anders.

0 Kudos
11 Replies
ArunJ_Intel
Moderator
2,241 Views

Hi


We are forwarding this case to subject matter experts. They will get back to you soon regarding this.


Thanks

Arun


0 Kudos
Mark_L_Intel1
Moderator
2,228 Views

Hi Anders,


Let me read the previous thread to understand more about this issue.


At the same time, do you happen to have any reproducer?


Mark


0 Kudos
cedronius__anders
New Contributor I
2,221 Views

 

Hi Mark,

setting these parameters in the GOP control would reproduce the GOP pattern mentioned above

GopPicSize = 24
GopRefDist = 3
IdrInterval = 0
GopOptFlag = MFX_GOP_CLOSED | MFX_GOP_STRICT

 However if you are unable to set those parameters yourself I can put together some C++ project that does that. Just let me know.

 

/A 

0 Kudos
Mark_L_Intel1
Moderator
2,201 Views

Thanks Anders,


It would be better if you can create a reproducer, I can do it but it will cost time and also if my reproducer has an error, it would cost more time to communicate.


If we do it with sample_encode, what would be the sample_encode command line and input video?


Mark


0 Kudos
cedronius__anders
New Contributor I
2,190 Views

No problem,

Give me a day or two and I'll put something together.

/Anders.

0 Kudos
Mark_L_Intel1
Moderator
2,177 Views

Thanks Anders,


If it takes too much time, let me know and we can work together.


Mark


0 Kudos
cedronius__anders
New Contributor I
2,167 Views

OK made a project that illustrates the 'problem' in a simple way.

https://github.com/Unit-X/qsv_enc_lab

 

See the source file for details but as I write ->

 

//Reproducing the out of GOP backward referencing pictures add MFX_GOP_STRICT to the GopOptFlag

// Without MFX_GOP_STRICT the output.h264 produces ->

/*
FRAME type B PTS: 84600 DTS 84600 Frame number: 49
FRAME type P PTS: 88200 DTS 86400 Frame number: 50
FRAME type IDR PTS: 90000 DTS 88200 Frame number: 51
FRAME type P PTS: 95400 DTS 90000 Frame number: 52
FRAME type B PTS: 91800 DTS 91800 Frame number: 53
FRAME type B PTS: 93600 DTS 93600 Frame number: 54
*/

// With MFX_GOP_STRICT set the output.h264 produces ->

/*
FRAME type B PTS: 82800 DTS 82800 Frame number: 48
FRAME type B PTS: 84600 DTS 84600 Frame number: 49
FRAME type IDR PTS: 90000 DTS 86400 Frame number: 50
FRAME type B PTS: 88200 DTS 88200 Frame number: 51
FRAME type P PTS: 95400 DTS 90000 Frame number: 52
*/
0 Kudos
Mark_L_Intel1
Moderator
2,159 Views

Thanks so much and really appreciated.


Let me try it on my side and If confirmed, I will submitted a bug. Will keep you updated.


Mark Liu


0 Kudos
Mark_L_Intel1
Moderator
2,121 Views

Hi Anders,


Yes, I tried your code and I can reproduce the behavior you described.


I will send this to dev team for investigation, for now, I am not sure if this is expected or a bug, I will keep you updated.


Just 2 more question:

  • For the input yuv file, is it a normal video and just duplicate all the static image from the first frame?
  • I checked the source file and didn't find anything special for encoding configuration, can you confirm?


Mark


0 Kudos
cedronius__anders
New Contributor I
2,115 Views

Hi Mark,

 

Yes that is correct I just feed the same image to the encoder over and over.

And regarding the encoder configuration, it's taken from intels simple example encoder and the parameters are the same. The addition is the GOP structure parameters. 

 

Talk to you later and will be great to hear what they say. 

 

/Anders

 

0 Kudos
Pamela_H_Intel
Moderator
1,177 Views

Apologies for the delays. Media SDK functionality has been superseded by oneVPL, FFmpeg and GStreamer.

 

For more information see the oneVPL Landing Page: https://www.intel.com/content/www/us/en/developer/tools/oneapi/onevpl.html where you will find, among other information:

 

oneVPL version 2.x is an API that is the successor to Intel® Media SDK. The Intel Media SDK project is no longer active and will not be updated. Packages and downloads are still available on GitHub*. For continued support and access to new features, Intel Media SDK users are encouraged to move to oneVPL as soon as possible.

·       Upgrade from Intel Media SDK to oneVPL

·       Intel Media SDK Discontinuation Notice

·       Download from GitHub

 

Let us know if you have any questions about oneVPL or how to convert from MediaSDK.


0 Kudos
Reply