Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

H.264 encoder key frame generation behavior

froueint
Beginner
1,046 Views

Hi,

I have noticed that the H.264 encoder doesn't handle the transition from a still scene to a fast motion scene very well. In particular, the encoder doesn't send an IDR frame when we go to a fast motion scene and this results in one of two behaviors:

1) In CBR/VBR: You see a lot of artifacting and it looks very bad
2) In AVBR: The video looks good, however, at the point of transition to fast motion, the encoder generates several very large P frames (this is a problem for us since we are streaming)

It seems to me that when the transition to fast motion happens the encoder should simply generate an IDR frame. It looks like it is trying to follow the GOP size very strictly and only issues an IDR frame at the beginning of the next GOP. I'm using 150 for GOP size, GopRefDist of 1 and IdrInternval of 0; and I'm not setting the MFX_GOP_STRICT flag.

Any help would be appreciated.

Regards,
Farhad

0 Kudos
8 Replies
Petter_L_Intel
Employee
1,046 Views
Hi Farhad, Media SDK does not support automatic I frame insertion in case of increased motion. The encoder will follow the GOP pattern configured during initialization or reset. However, IDR frame can be inserted manually at any time, decision on when to do this and based on what criteria is up the the developer. If per frame quality control is of importance then you may want to explore the CQP rate control mode. That said, not sure it will fit well your streaming frame size limitations. Regards, Petter
0 Kudos
froueint
Beginner
1,046 Views
Thanks Petter. I actually tested this case (still image to fast motion) with x264 and it didn't generate an IDR frame either; however, it didn't generate very large frames and the quality didn't suffer either. Could you look into improving the behavior of the encoder in this case for future releases? Regards, Farhad
0 Kudos
Petter_L_Intel
Employee
1,046 Views
Hi Farhad, CQP work the same way as the other rate control modes, that is, I-frames are inserted based on configured GOP pattern. We will discuss potential future feature addition to support automatic I-frame insertion on scenes with heavy movement. Near term, such a feature is unlikely but we will assess feasibility of this for future releases. Regards, Petter
0 Kudos
Yabo_W_
Beginner
1,046 Views

Hi Petter,

I notice that all the gop size are the same. When do you plan to support automatic I-frame insertion?

Regards,
Yabo

0 Kudos
Sravanthi_K_Intel
1,046 Views

Hi Yabo,

Currently the SDK does not support automatic I-frame insertion.

But - if you go back to the message thread, the issue can be traced back to controlling bitrates when streaming (especially when the scene change is big). If you are looking at similar issues and a solution, this information may help you:

The CBR and VBR methods would not adhere to the required bitrate to conform to HDR during big scene changes, thus resulting in occasional spikes in encoded frame sizes. Since then, the SDK has been trying to add more bitrate control methods for different use cases, and one such method is Look Ahead. You can find more information about it here - https://software.intel.com/en-us/articles/common-bitrate-control-methods-in-intel-media-sdk

Quoting from the article - " It distributes available bit budget between frames to produce the best possible encoding quality. This method is not a HRD compliant. It generates good results on the complex data such as fast motion video, computer generated sequences."

ALSO - Can you please share any high scene change video you may be using for your development? We could use customer provided videos as test cases, which will be helpful to both.

 

0 Kudos
lee_j_
Beginner
1,046 Views
Hi Petter

I would like to know whether the current frame is a key frame in the encoding process.
How do you know?

Regards,
jmlee

0 Kudos
Mark_L_Intel1
Moderator
1,046 Views

Hi Lee,

Could you more specific to the question? Which algorithm did you refer to?

Mark

0 Kudos
lee_j_
Beginner
1,046 Views

Hi Mark.

If you do the following:
sts = m_pmfxENC-> EncodeFrameAsync (NULL, & m_pEncSurfaces [nEncSurfIdx], & pCurrentTask-> mfxBS, & pCurrentTask-> EncSyncP);

How do I know if I am on the current frame is a key frame?

Or Do I know from another function?

Regards,

Lee

0 Kudos
Reply