Key Frame (I/IDR) insertion

[Frantisek_K_]

Hello,

I'm trying to implement key frame insertion into my video encoding process. However, I'm facing issues with it.

When calling function MFXVideoENCODE_EncodeFrameAsync (&gop_ctrl, ...), I provide on requested key-frame in structure gop_ctrl.FrameType = FRAME_I | FRAME_IDR | FRAME_REF.

However, I'm not getting I-frame on the expected position ! (Or not at all).
(Please note I'm using code similar to sample_encode or sample_videoconf)

Other parameters in mfxInfoMFX structure which I'm using, are:

VBR encoding, NumSlice=4, NumRefFrame=0 (or 1), EncodedOrder=0, GopPicSize=24, IdrInterval=0, GopOptFlag=0, GopRefDist=1 or 4

I observed the following:

1. when using SW library, no extra key frame is inserted - never !!!
2. when using HW library, and GopRefDist=1 (i.e. no B-frames), then Key Frame is inserted in expected position
3. when using HW library, and GopRefDist=4, the Key Frame is inserted but on wrong position (shifted few frames earlier)

Could you please advice what else should I check? Which other parameters may influence key frame insertion process?

Or is there something wrong with Key Frame Insetion (known bug in Intel Media SDK R2)?
It looks to me like KeyFrame Insertion works only for P frames and only in HW library. That would be a pity!

Thanks

Frank

[Anthony_P_Intel]

Hi Frank,

I believe the Media SDK implementation may be optimizing your GOP structure based on what it feels is best, and to force the insertion you will need to include MFX_GOP_STRICT in your code.

Please see the "Forced Key Frames" section of this doc:  http://software.intel.com/en-us/articles/video-conferencing-features-of-intel-media-software-development-kit/

[Frantisek_K_]

Hello,

unfortunately setting MFX_GOP_STRICT has no impact on key frame insertion.
I performed the above mentioned test with setting MFX_GOP_STRICT but result is the same:

1. when using SW library, no extra key frame is inserted - not at all !
2. when using HW library, then Key Frame is inserted on expected position only when GopRefDist=1,

This still leads me to think there is some bug/gap in the core SDK routines... :(
(unless I need to do something else than only setting MFXVideoENCODE_EncodeFrameAsync (&gop_ctrl, ...) on proper frame)

Remark:
GopRefDist=1 means no B-frames = bad efficiency of video compression.

Btw. the article you have mentioned also uses only mfxInfoMFX::GopRefDist = 1  (for "optimal low latency").

This is valid for videoconferencing, but practically useless for effective HD video encoding (which is my aim, btw.).

"Optimization" of GOP structure is welcome for general HD video, that's right.
But if I want to insert a key frame, I want to do it on the exact position (scene cut), in such case I expect encoder to follow my instruction :)
Otherwise it makes no sense to define manually keyframes.

Frank

[Petter_L_Intel]

Hi Frank,

Using low latency configuration, like the encoder config you are using, immediate I-frame insertion is possible for both SW and HW mode. Just to make sure I just tried I-frame insertion locally for both HW and SW mode and it works fine.

Not sure why you're encountering issues with SW mode.  Could you please share your exact encoder configuration (you can capture this using the media SDK tracer tool) and also the version of Media SDK SW DLL (from 2013 R2?) you are using.

The behavior you describe for the case when GopRefDist > 1  is expected. Please see the chapter in the Media SDK reference manual which describes "Forced Key Frame Generation".

Regards,
Petter

[Frantisek_K_]

Hi Petter,

thanks for your response.

I'm using libraries:
HW: file version 4.13.11.22, product version 4.0.1533432.75720
SW: file version 4.13.6.21,  product version 4.0.0000760.60139  (from SDK 2013 R2)

I run and encoded four times the same video (275 frames), just changing HW vs. SW library and GopRefDist = 1 vs. 4 (hence 4 combinations),
all other parameters were identical.

I requested frames 9, 40 and 55 to be keyframes (IDR frames, counting from 0)

Resulting gop-structure is displayed in file *_gop.log, and associated trace analysis in *_analysis.log
As you can see, only HW library and GopRefDist=1 fulfilled the request.

SW library ignored forced keyframes completely.

HW library with GopRefDist=4 placed them to different positions (38 and 53), which is wrong.
Key frame must be placed to exactly requested position (scene cut), this is the reason and purpose why do we want to force keyframes :)
(And obviously, for HD video encoding B-frames are a necessity for achieving high quality with reasonable bitrate)

Any advice is welcome.

Thanks (and Happy New Year 2014 - additionally :) )

Frank

[Petter_L_Intel]

Hi Frank,

not sure what is going on with regards to your SW execution case. Locally, using the exact same configuration and SW DLL (from 2013 R2), I frames are inserted at the expected locations.  Can you please revisit your implementation to make sure Iframe/IDR/REF are inserted correctly in your code for the SW execution path.

Regarding the case when you're using B-frames. The behavior you observe is the expected one as stated in my earlier post.

You may be able to achieve your goal  by moving to "EncodedOrder" mode, which requires you to manually specify the exact frame type for each encoded frame.

Regards,
Petter

[Frantisek_K_]

Hi Petter

The actual encoding pipeline is identical both for HW and SW, basically it is practically identical code to "sample_encode" example
- modified for insertion of I/IDR frames as mentioned in the first post above.
Selection of HW/SW is done in the initializaton - all the rest of the code is identical.
So it is strange for me why SW library ignores it.

Btw. I noticed that SW and HW encoding for the same yuv-source and same parameters produce different .h264 streams, so internal encoding logic in HW and SW is apparently slightly different.
(HW is much faster but SW gives slightly better picture - I noticed in HW some macroblocking artefacts in case of too long GOP)

Regarding B-frames: It might be expected behavior, but honestly - it is not logical and useless for practical video-encoding :(
If I want to insert keyframe (=I/IDR-frame), I want to do it intentionally on some sharp scene cut to avoid unwanted artefacts (macroblocing).
This is the reason for forced key-frames.
And in such case encoder must place it on exactly this frame and adjust GOP structure of other surrounding frames accordingly.
Otherwise I would leave it completely on encoder and not use forcing key frames.

I'd suggest to Intel Media Development Team to reconsider it and change this behavior to "natural", logical way...
 

Thanks

Frank

[Petter_L_Intel]

Hi Frank,

SW and HW encoder are implemented in very different ways, and thus the resulting encoded stream does have different properties. This is the expected behavior.

We have no plans of changing the behavior for the case of manual insertion I-frames.  Please explore using "EncodedOrder" mode instead.

Regards,
Petter

[Eyal_E_]

Hi,

I understand that in display order mode, when B frames are enabled the behaviour of key frames insertion is inconsistent-

i.e. Placed key frames location in presentation order is different than the order mfxEncodeCtrl was used with EncodeFrameAsync.

From tests i have done i can see that there is a constant offset between  presentation order index of requested  vs placed.

The constant offset starts from the second key and on, (e.g. seen offset of 2), see below:

Keys are placed on presentation order [ 0, 46,94,142,..]

Instead of [0,48,96,144,..]

When i try to patch it to place on presentation order of +2 starting from the second key i get keys on:

[0,49,96,144,..]

Second Key (presentation order 49) has some issue but the rest are OK,

Can you please suggest a way to determine the required offset based on number of B frames (i.e. GopRefDist) perhaps depending on other Gop params?

Thanks,

Eyal E

[Sravanthi_K_Intel]

Hi Eyal - You are starting a new thread on an old one. Can you kindly start a new thread and refer to this thread if needed? Your post can be easily lost if you pile on to older threads.

When you post your question again, can you please help with some more details on the system you are on (OS, MSS version)? Also, did you use an existing sample or tutorial and modified them to add your changes wrt key frame insertion? Can you please send us the code snippet? Meantime, I will get back if I have any simple information on this. 

[lee_j_]

Hi

If you do the following:
sts = m_pmfxENC-> EncodeFrameAsync (NULL, & m_pEncSurfaces [nEncSurfIdx], & pCurrentTask-> mfxBS, & pCurrentTask-> EncSyncP);

How do I know if I am on the current frame is a key frame?

Or Do I know from another function?

Regards,

Lee