DirectShow h264 Encoder KeyFrames only and then Decode KeyFrames only

[dashut]

I have IPP7.0 and Media SDK 2.0.12.24071.
Im working with the "sample_filters.sln", there I'm interested in "h264_enc_filter" and "h264_dec_filter".
Using these filters in GraphEdit: I manage to connect them and run them:
First I encode (using the h264_enc_filter) and then I decode (using the h264_dec_filter).
So that works for sure (although it requires another filter added before the encoder to convert the sample type of the input: but that's not what I want to ask here).

Now what I want to do is: I want to save/play ONLY the KeyFrame samples of the h264_enc_filter, and then decode using h264_dec_filter only the KeyFrames I've saved.
I don't mind that the Encoder will encode everything, but then I'll only send the KeyFrames to the Decoder and it will play KeyFrames only.

How can this be done?
How do I identify the KeyFrames in the h264_enc_filter?

Just to to be clear: by saying "KeyFrame" I mean the first sample of the GOP (not including the delta frames). So if the interval of KeyFrames is set to 50 and my source is PAL (25 frames per second): then I expect a KeyFrame every 2 seconds.
[added:] When using Microsoft WMV3, you can set the KeyFrame to be 50 (for example). So after I Encode with WMV3, I can either send to the Decoder (usually "The Player") the entire stream (whole GOP: KeyFrame + delta's) or just the KeyFrame (which is one every 2 seconds or 50 frames for PAL which has 25 frames per second) - and the Decoder would handle it! The advantage of Encoding everything but then either sending the entire stream or just the KeyFrame: is good for low bandwidth connections where you want to stream only the KeyFrames. Or you want to stream only the KeyFrames to Play "super fast".

Any help on this will be greatly appreciated.

[Petter_L_Intel]

Hi dashut,

mfxBitstream.FrameType provides info about the encoded frame type.This information is available in the generated bitstream after successful SyncOperation (base_encoder.cpp)

Based on the solution you describe I hope the above information is sufficient.

Regards,

Petter

[dashut]

Here's what I found out: so please correct me if I'm wrong:

1. In: "struct PSControl" member: "i_frame_internal" determines how many frames(/samples) will a KeyFrame occur. The default in the code was 200, so if I want a KeyFrame every 50 frames (every 2 seconds for PAL which has 25 frames per second): then I have to set this value to 50 (default params are set in file: "h264_enc_filter.cpp" function: "CH264EncVideoFilter::SetDefaultParams(void)" line 41: "m_H264Params.ps_control.i_frame_internal = 200;" which I changed to 50).

2. In order to determine if the Encoder produced a KeyFrame or not I did the following in file: "mfx_video_enc_filter.cpp" function: "HRESULT CEncVideoFilter::DeliverBitstream(mfxBitstream* pBS)":

BOOL bSync = pBS->FrameType & MFX_FRAMETYPE_I;
hr = pOutSample->SetSyncPoint(bSync);

3. I delivered the output of this Encoder to the h264 Decoder and everything worked as expect.

4. Then I made another change in function: "HRESULT CEncVideoFilter::DeliverBitstream(mfxBitstream* pBS)" and sent ONLY the KeyFrames by adding an "if" statement:

if (bSync) // bSync is calculated: pBS->FrameType & MFX_FRAMETYPE_I
hr = m_pOutput->Deliver(pOutSample);

and I Decoded just that and it worked! I saw only KeyFrame samples (meaning: a frame every 2 seconds - which is exactly what I wanted to achieve).

Now the question is: although I see the results that I wanted: is this the correct approach?

I also noticed a huge delay from the Decoder which is explained in thread: http://software.intel.com/en-us/forums/showthread.php?t=86910 (thread topic: "Problem to decode H264 video in real-time").

So to reduce this delay and have the Decoder release a sample/frame for every sample/frame given to it I have to use SDK 3.0?

Thanks, Dashut.

[dashut]

Here's what I found out: so please correct me if I'm wrong:

1. In: "struct PSControl" member: "i_frame_internal" determines how many frames(/samples) will a KeyFrame occur. The default in the code was 200, so if I want a KeyFrame every 50 frames (every 2 seconds for PAL which has 25 frames per second): then I have to set this value to 50 (default params are set in file: "h264_enc_filter.cpp" function: "CH264EncVideoFilter::SetDefaultParams(void)" line 41: "m_H264Params.ps_control.i_frame_internal = 200;" which I changed to 50).

2. In order to determine if the Encoder produced a KeyFrame or not I did the following in file: "mfx_video_enc_filter.cpp" function: "HRESULT CEncVideoFilter::DeliverBitstream(mfxBitstream* pBS)":

BOOL bSync = pBS->FrameType & MFX_FRAMETYPE_I;
hr = pOutSample->SetSyncPoint(bSync);

3. I delivered the output of this Encoder to the h264 Decoder and everything worked as expect.

4. Then I made another change in function: "HRESULT CEncVideoFilter::DeliverBitstream(mfxBitstream* pBS)" and sent ONLY the KeyFrames by adding an "if" statement:

if (bSync) // bSync is calculated: pBS->FrameType & MFX_FRAMETYPE_I
hr = m_pOutput->Deliver(pOutSample);

and I Decoded just that and it worked! I saw only KeyFrame samples (meaning: a frame every 2 seconds - which is exactly what I wanted to achieve).

Now the question is: although I see the results that I wanted: is this the correct approach?

I also noticed a huge delay from the Decoder which is explained in thread: http://software.intel.com/en-us/forums/showthread.php?t=86910 (thread topic: "Problem to decode H264 video in real-time").

So to reduce this delay and have the Decoder release a sample/frame for every sample/frame given to it I have to use SDK 3.0?

Thanks, Dashut.

[Petter_L_Intel]

Hi Dashut,

1. Correct, you can control GOP size with this parameter. Please note that for the 3.0 (2012) samples this filter parameter has been renamed to GopPicSize to correspond with the actual encoder parameter name.

2-4. It looks like this would be a working approach, especially since it seems to work as you expected. I do not have a similar implementation so I cannot provide further details.

As you point out, Media SDK 2.0 was never intended to be used in low latency scenarios. Media SDK 3.0 (2012) is bridging this gap. There are several other threads on this forum where this topic has been discussed, including how to configure the Media SDK components for low latency.
Also, the new sample filters have specific presets to enable low latency mode.

Regards,

Petter