H.264 profile support

[salsailor]

Does the 2012 release H.264 encoder/decoder support Constrained Baseline Profile? The profile settingsheader filehas Baseline only.

thanks for the timely release of this sdk!

Lee

[Eric_S_Intel]

Hi Lee,

The Media SDK does not have a manual way to specify Constrained Baseline Profile. It can however be done by setting the constraint_set1_flag to 1 in the SPS, and the Media SDK does have the ability to insert custom SPS/PPS data into the bitstream. Take a look at mfxExtCodingOptionSPSPPS in the reference manual, and let us know if you need assistance implementing it.

Thanks

-Eric

[salsailor]

Eric,

Thanks for the reply. I'll try something like below, is it about right? If I only need to change sps, I should set pps to NULL? Or should I call a GetParam first then reassign the VideoParam?

Lee


// set sps stream buffer
u8 sps_buffer[] = {0x00, ....} // constraint_set1_flag to 1
uint sps_buffer_length = sizeof(sps_buffer);

/* configure mfxExtCodingOptionSPSPPS */

memset(&option,0,sizeof(option));

option.Header.BufferId=MFX_EXTBUFF_CODING_OPTION_SPSPPS;

option.Header.BufferSz=sizeof(option);

option.SPSBuffer=sps_buffer;

option.SPSBufSize=sps_buffer_length;

option.PPSBuffer=NULL;

option.PPSBufSize=0;

/* configure mfxVideoParam */

mfxVideoParam param;

param.NumExtParam=1;

option_array=&option;

param.ExtParam=&option_array;

/* encoder initialization */

mfxStatus status;

status=MFXVideoENCODE_Init(session, &param);

[Eric_S_Intel]

Hi Lee,

Our team is trying to continually improve the Media SDKDevelopers guide with each new release. Luckily, Petter has started documenting this method. I am attaching our rough verbiage for the next manual. Hopefully, it will help. Thanks Petter!

4.13 Custom control of encoded AVC SPS/PPS data

Intel Media SDK provides control of a subset of SPS/PPS header parameters defined by the AVC standard.

For control of specifc SPS/PPS parameters that cannot be controlled via the SDK API, please refer to the following method.

1) Encode an AVC stream with a set of parameters closest matching the desired SPS/PPS set
2) Use 3rd party tool such as Elecard* Stream Analyzer to extract the SPS and/or PPS header
3) Create byte array in the application that representing the extacted SPS/PPS header
4) Make the specific modifications to SPS and/or PPS header
5) To encode a stream using the customs SPS/PPS, attach the mfxExtCodingOptionSPSPPS structure (refrencing the custom header) to mfxVideoParam as an extended buffer

Note that, when using this method the SPS parameters set via mfxVideoParams will be overwritten by the custom SPS buffer.

As an example, to be fully compliant with H.264 Constrained Baseline profile, the SPS constraint_set1_flag should be set to 1. It is not possible to achieve this via the SDK API so manual control of SPS is required. See code example below, and please refer to the AVC standard for details.

[bash]// Example SPS header extracted from encoded stream

// (blue-marked byte represents constrained_set settings)

unsigned char custom_sps[] = {

0x00, 0x00, 0x00, 0x01, 0x27, 0x42, 0x40, 0x1F,
0x95, 0xB0, 0x14, 0x01, 0x6E, 0xC0, 0x44, 0x00,
0x00, 0x03, 0x00, 0x04, 0x00, 0x00, 0x03, 0x00,
0xF3, 0x81, 0x00, 0x00, 0xC5, 0x35, 0x00, 0x00, 
0x20, 0x0B, 0x26, 0xF7, 0xBE, 0x0E, 0xD0, 0x44, 
0x23, 0x70 };

mfxExtCodingOptionSPSPPS m_extSPSPPS;
MSDK_ZERO_MEMORY(m_extSPSPPS);

m_extSPSPPS.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_SPSPPS;
m_extSPSPPS.Header.BufferSz = sizeof(mfxExtCodingOptionSPSPPS);
m_extSPSPPS.SPSBuffer = custom_sps;
m_extSPSPPS.SPSBufSize = sizeof(custom_sps);

// Add SPSPPS extended byffer to encoder parameter extended buffer list
m_EncExtParams.push_back((mfxExtBuffer *)&m_extSPSPPS);
[/bash]

As an alternative, a developer may instead do manual parsing and patching of the bit stream that the encoder delivers.

[salsailor]

Thanks again for the reply!

But if the custom sps buffer is just going to overwrite the wholesps buffer used by the encoder, then it will onlywork if the video stream is one consistent stream, i.e. frame size, frame rate does not change.Otherwise, there's no way I can construct the custom sps buffer during runtime.

Unless there is a method (callback) to passback the sps buffer to be used by the encoder then let the user overwrites it only the bitsneeds to be changed justbefore the encoding start, i.e. after initialization of a session ??

Lee

[Eric_S_Intel]

Right, I hear what you are saying. The support we have is limited as best. Let me discuss this with my colleges and see if we can come up with a more dynamic solution.

Eric

[salsailor]

Ok. Thanks for looking into the possibilities. Meanwhile, I think I can explore changing the sps at the downstream receiver of the encoded stream by setting the constraint_set0_flag.

But will it be safe to assume that by setting profile to BP using the sdk, the Intel encoder pretty much is CBP except for the flag mentioned above in the sps. i.e. no advanced feature like FMO or ASO is used or it depends on the version of encoder/hardware at runtime?

Lee

[Eric_S_Intel]

Hi Lee,

Thanks for your patience. The Media SDK 2012 does have the ability to retrieve the SPS/PPS after the encoder has been initialized. Heres the sample code:

[bash]mfxVideoParam par;

    MSDK_ZERO_MEMORY(par);


    mfxExtCodingOptionSPSPPS opt;

    MSDK_ZERO_MEMORY(opt);


    opt.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_SPSPPS;

    opt.Header.BufferSz = sizeof(mfxExtCodingOptionSPSPPS);

    par.mfx.CodecId = MFX_CODEC_AVC;

    par.ExtParam = new mfxExtBuffer *[1];

    par.ExtParam[0] = (mfxExtBuffer*)&opt;

    par.NumExtParam = 1;

    opt.SPSBuffer = new mfxU8 [100];

    opt.PPSBuffer = new mfxU8 [100];

    opt.PPSBufSize = 100;

    opt.SPSBufSize = 100;

    sts = m_pmfxENC->GetVideoParam(&par);

    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

[/bash]

The returned SPS buffer will point to the Nal_Unit() containing the SPS data. You will need to parse the Nal_unit() header and seq_parameter_set_data(), and set the constraint_set1 bit. Once the data is modified, you can reset the encoder with the new values.

Granted, its a quite an endeavor to set 1 bit. Ill feed my info back to the architects and see if we can make it more streamlined in the future.

Thanks

-Eric

[salsailor]

Eric,

Thanks for getting back to me. I'll give it a shot using this method.

But on the subject of BP and CBP compatible issues, if I set the constrained bit using above method, the Intel encoder is not going to use any of the advanced features that is not supported by CBP, i.e. FMO, ASO and RS? Otherwise, a CBP decoder will have problems with the stream.

Please confirm.

Lee

[Eric_S_Intel]

Hi Lee,

I checked with the architects and can confirm that if you set the constraint_set1 flag, the encoder will refrain from using the advanced features like FMO, ASO, and RS. It will produce a constrained stream.

Thanks

-Eric

[salsailor]

Hi Eric,

Thanks for the support. I'm trying to see if I can get some code to parse sps header first and let me know if Intel can quickly update the sdk to support cbp.

But meanwhile I have another problem:

When I set the Profile to BP(66) and Level to 31, MFXVideoEncode_Query(mfxVideoParam, ...) always returns "incompatible parameter" and will reset the level to 40. Is this checking depends on other parameters ? Here is what I passed in mfxVideoParam, which one is incompatible with the profile level?

-(m_VideoParam).mfx{reserved=0x0241fd38 reserved4=0 BRCParamMultiplier=1 ...}mfxInfoMFX
+reserved0x0241fd38unsigned int [7]
reserved40unsigned short
BRCParamMultiplier1unsigned short
-FrameInfo{reserved=0x0241fd58 FrameId={...} FourCC=842094158 ...}mfxFrameInfo
+reserved0x0241fd58unsigned int [6]
+FrameId{TemporalId=0 PriorityId=0 DependencyId=0 ...}mfxFrameId
FourCC842094158unsigned int
Width1280unsigned short
Height736unsigned short
CropX0unsigned short
CropY0unsigned short
CropW1280unsigned short
CropH720unsigned short
FrameRateExtN30unsigned int
FrameRateExtD1unsigned int
reserved30unsigned short
AspectRatioW1unsigned short
AspectRatioH1unsigned short
PicStruct1unsigned short
ChromaFormat1unsigned short
reserved20unsigned short
CodecId541283905unsigned int
CodecProfile66unsigned short
CodecLevel31unsigned short
NumThread0unsigned short
TargetUsage4unsigned short
GopPicSize200unsigned short
GopRefDist1unsigned short
GopOptFlag0unsigned short
IdrInterval0unsigned short
RateControlMethod2unsigned short
InitialDelayInKB0unsigned short
QPI0unsigned short
Accuracy0unsigned short
BufferSizeInKB3750unsigned short
TargetKbps4800unsigned short
QPP4800unsigned short
MaxKbps4800unsigned short
QPB4800unsigned short
Convergence4800unsigned short
NumSlice1unsigned short
NumRefFrame1unsigned short
EncodedOrder0unsigned short
DecodedOrder4unsigned short
ExtendedPicStruct200unsigned short
TimeStampCalc1unsigned short
+reserved20x0241fdac ""unsigned short [10]
JPEGChromaFormat4unsigned short
Rotation200unsigned short
+reserved30x0241fdaa "
"unsigned short [11]

[Eric_S_Intel]

Hi Lee,

First, make sure you set the CodecId on the output struct. If you dont, Query automatically returns Invalid Param. Given you said the level was being adjusted to 40, you are probably doing that.

I debugged your params and noticed the code adjusting your level due to the value of BufferSizeInKB. Try setting this to zero and letting the MSDK figure out the right buffer size for the level. I did that and it worked fine.

Thanks

Eric

[salsailor]

Hi Eric,

The buffer size does matter. I set it to 0, then the level size came up to 32 instead of 31. I think the framesize is also an issue. if you look at my frame input size it is 1280x736 (vs. 1280x720 used by the crop size). Once I shrink it a bit (round it down the input size) using 1280x704. It will accept level 31.

I don't know if this is a bug or not, but shouldn't the encoder look at the output size (crop size) vs. the input frame size to decide the level it can support?

Lee