H.264 encoder key frame generation behavior

[froueint]

Hi,

I have noticed that the H.264 encoder doesn't handle the transition from a still scene to a fast motion scene very well. In particular, the encoder doesn't send an IDR frame when we go to a fast motion scene and this results in one of two behaviors:

1) In CBR/VBR: You see a lot of artifacting and it looks very bad
2) In AVBR: The video looks good, however, at the point of transition to fast motion, the encoder generates several very large P frames (this is a problem for us since we are streaming)

It seems to me that when the transition to fast motion happens the encoder should simply generate an IDR frame. It looks like it is trying to follow the GOP size very strictly and only issues an IDR frame at the beginning of the next GOP. I'm using 150 for GOP size, GopRefDist of 1 and IdrInternval of 0; and I'm not setting the MFX_GOP_STRICT flag.

Any help would be appreciated.

Regards,
Farhad

[Petter_L_Intel]

Hi Farhad, Media SDK does not support automatic I frame insertion in case of increased motion. The encoder will follow the GOP pattern configured during initialization or reset. However, IDR frame can be inserted manually at any time, decision on when to do this and based on what criteria is up the the developer. If per frame quality control is of importance then you may want to explore the CQP rate control mode. That said, not sure it will fit well your streaming frame size limitations. Regards, Petter

[froueint]

Thanks Petter. I actually tested this case (still image to fast motion) with x264 and it didn't generate an IDR frame either; however, it didn't generate very large frames and the quality didn't suffer either. Could you look into improving the behavior of the encoder in this case for future releases? Regards, Farhad

[Petter_L_Intel]

Hi Farhad, CQP work the same way as the other rate control modes, that is, I-frames are inserted based on configured GOP pattern. We will discuss potential future feature addition to support automatic I-frame insertion on scenes with heavy movement. Near term, such a feature is unlikely but we will assess feasibility of this for future releases. Regards, Petter

[Yabo_W_]

Hi Petter,

I notice that all the gop size are the same. When do you plan to support automatic I-frame insertion?

Regards,
Yabo

[Sravanthi_K_Intel]

Hi Yabo,

Currently the SDK does not support automatic I-frame insertion.

But - if you go back to the message thread, the issue can be traced back to controlling bitrates when streaming (especially when the scene change is big). If you are looking at similar issues and a solution, this information may help you:

The CBR and VBR methods would not adhere to the required bitrate to conform to HDR during big scene changes, thus resulting in occasional spikes in encoded frame sizes. Since then, the SDK has been trying to add more bitrate control methods for different use cases, and one such method is Look Ahead. You can find more information about it here - https://software.intel.com/en-us/articles/common-bitrate-control-methods-in-intel-media-sdk

Quoting from the article - " It distributes available bit budget between frames to produce the best possible encoding quality. This method is not a HRD compliant. It generates good results on the complex data such as fast motion video, computer generated sequences."

ALSO - Can you please share any high scene change video you may be using for your development? We could use customer provided videos as test cases, which will be helpful to both.

[lee_j_]

Hi Petter

I would like to know whether the current frame is a key frame in the encoding process.
How do you know?

Regards,
jmlee

[Mark_L_Intel1]

Hi Lee,

Could you more specific to the question? Which algorithm did you refer to?

Mark

[lee_j_]

Hi Mark.

If you do the following:
sts = m_pmfxENC-> EncodeFrameAsync (NULL, & m_pEncSurfaces [nEncSurfIdx], & pCurrentTask-> mfxBS, & pCurrentTask-> EncSyncP);

How do I know if I am on the current frame is a key frame?

Or Do I know from another function?

Regards,

Lee