Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

What kind of SPS parameter does msdk h264 encoder support?

Zheng_L_
Beginner
953 Views

I init the h264 encoder with a specified SPS/PPS parameter from an existing h264 stream,below is the info of sps/pps reported by Elecard StreamEye software:

spspps.JPG

I attach the sps/pps parameter to mfxVideoParam and call MFXVideoEncode_Query(),the function return MFX_ERR_NONE,but when I call MFXVideoENCODE_Init() with the same mfxVideoParam parameter, the function return MFX_ERR_INCOMPATIBLE_VIDEO_PARAM,why Query() is OK,but Init() failure? what kind SPS parameter does h264 encoder support?(I had checked pps parameter is ok)

 

0 Kudos
11 Replies
Zheng_L_
Beginner
953 Views

who can help me?
 

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zeng,

Please refer to mediasdk-man.pdf  and look for mfxExtCodingOptionSPSPPS structure on page 97. There are set  SPS header parameters supported by h.264 encoder. To extract raw SPS/PPS header info from the encoded stream, you should call MFXVideoENCODE_GetVideoParam with mfxExtCodingOptionSPSPPS structure attached to the mfxVideoParam structure and  basic mediasdk workflow is after setting of parameters is call Init() ->Query() function. Also a note, make sure you are allocating sufficient space for SPS/PPS buffer. I hope this is what you are looking for.

Thanks,

 

 

0 Kudos
Zheng_L_
Beginner
953 Views

Harsh Jain (Intel) wrote:

Hi Zeng,

Please refer to mediasdk-man.pdf  and look for mfxExtCodingOptionSPSPPS structure on page 97. There are set  SPS header parameters supported by h.264 encoder. To extract raw SPS/PPS header info from the encoded stream, you should call MFXVideoENCODE_GetVideoParam with mfxExtCodingOptionSPSPPS structure attached to the mfxVideoParam structure and  basic mediasdk workflow is after setting of parameters is call Init() ->Query() function. Also a note, make sure you are allocating sufficient space for SPS/PPS buffer. I hope this is what you are looking for.

Thanks,

Hi Harsh:

    I extrack the raw SPS/PPS parameters from MFXVideoDECODE_DecodeHeader(),not MFXVideoENCODE_GetVideoParam() because the encoder is not initialized,I attach the mfxExtCodingOptionSPSPPS struct to the mfxVideoParam parameter and call Init(),but the function return MFX_ERR_INCOMPATIBLE_VIDEO_PARAM, where is the problem? is the sps parameter is wrong,or incompatible with msdk encoder?

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zeng,

Ah, I see you are trying multiple-segment encoding? MSDK support all of SPS header parameters defined by the AVC standard. Encode_Init() returning MFX_ERR_INCOMPATIBLE_VIDEO_PARAM only mean it was unable to achieve compatibility and may be due to incompatible video parameters detected. Can you share your application code snippet with bitstream?  

Thanks,

0 Kudos
Zheng_L_
Beginner
953 Views

Hi,Harsh,

What is multi-segment encoding? my app just reencode every idr frame fron an existing stream and rewrite the reencoded idr frame to original stream,so I must force the encoder use a special sps/pps parameters;

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zheng,

Multi-segment encoding can be defined as dividing an input sequence of frames into segments and encoding them in different encoding sessions with same or different parameter sets. Please find more details in "media-sdk manual" on page no. 200 & 201.

MSDK API allows direct API control from API 1.4 and manual SPS control is not required. But if you would like to force any SPS/PPS parameters please use the following method:

1)Configure encoder with a set of parameters closest matching the desired SPS/PPS set.
2) After initalizing the encoder call GetVideoParam() with extended buffer mfxExtCodingOptionSPSPPS to extract the SPS and PPS selected by the encoder (make sure to allocate sufficient space for storage of SPS and PPS buffer)
3) Make the desired modifications to SPS and/or PPS buffer.
4) Apply the changes to the encoder by calling Reset(), referencing the new mfxVideoParam structure including the mfxExtCodingOptionSPSPPS extended buffer. Note that, when using this method the SPS/PPS parameters set via mfxVideoParams will be overwritten by the custom SPS/PPS buffer. 

Following is a example on how to manually control SPS "constraint_set" values. The same approach can be used to control any SPS or PPS parameters manually. 

// ... Encoder initialized
mfxExtCodingOptionSPSPPS m_extSPSPPS;
MSDK_ZERO_MEMORY(m_extSPSPPS);
m_extSPSPPS.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_SPSPPS;
m_extSPSPPS.Header.BufferSz = sizeof(mfxExtCodingOptionSPSPPS);
// Allocate sufficient space for SPS/PPS buffers (100 is enough)
m_extSPSPPS.PPSBuffer = (mfxU8*)malloc(100);
m_extSPSPPS.SPSBuffer = (mfxU8*)malloc(100);
m_extSPSPPS.PPSBufSize = 100;
m_extSPSPPS.SPSBufSize = 100;
// Add SPSPPS extended buffer to encoder parameter extended buffer list
// std::vector<mfxExtBuffer*> m_EncExtParams
// mfxVideoParam m_mfxEncParams
m_EncExtParams.push_back((mfxExtBuffer *)&m_extSPSPPS);
m_mfxEncParams.ExtParam = &m_EncExtParams[0];
m_mfxEncParams.NumExtParam = (mfxU16)m_EncExtParams.size();
// Retrieve selected encoder parameters
mfxStatus sts = m_pmfxENC->GetVideoParam(&m_mfxEncParams);
// Very simple representation of first set of SPS buffer members
typedef struct {
unsigned char NAL_bytes[5];
unsigned char SPS_Profile;
unsigned char SPS_Constraint_Set;
unsigned char SPS_Level;
// ... rest of SPS is ignored (needs parsing)
} BasicSPS;
BasicSPS* mySPS = (BasicSPS*) m_extSPSPPS.SPSBuffer;
mySPS->SPS_Constraint_Set |= 0x40; // Set constraint_set1 to 1
// Reset encoder with modified SPS buffer
sts = m_pmfxENC->Reset(&m_mfxEncParams);

Thanks, 

0 Kudos
Zheng_L_
Beginner
953 Views

Hi Harsh,

Thanks your reply,I follow your guide and modify my code to specify sps parameters,but I still failed,the Reset() function return  MFX_ERR_INCOMPATIBLE_VIDEO_PARAM,below is the code I modified:

    m_pMfxENC = new MFXVideoENCODE(m_pMfxSession->m_session);
    sts = m_pMfxENC->Init(&encPar);  // it is ok
    Ignore_MST_PartiaAcceleration(sts);
    assert(sts == MFX_ERR_NONE);
    if(sts != MFX_ERR_NONE)
        return FALSE;

{
        BYTE * spsBuffer = new BYTE[1024];
        BYTE * ppsBuffer = new BYTE[1024];
        mfxExtCodingOptionSPSPPS extSPSPPS;
        extSPSPPS.Header.BufferId = MFX_EXTBUFF_CODING_OPTION_SPSPPS;
        extSPSPPS.Header.BufferSz = sizeof(mfxExtCodingOptionSPSPPS);
        extSPSPPS.SPSBuffer = spsBuffer;
        extSPSPPS.SPSBufSize = 1024;
        extSPSPPS.PPSBuffer = ppsBuffer;
        extSPSPPS.PPSBufSize = 1024;
        extSPSPPS.SPSId = 0;
        extSPSPPS.PPSId = 0;
        mfxExtBuffer * extBuffer = (mfxExtBuffer *)&extSPSPPS;
        encPar.ExtParam = &extBuffer;
        encPar.NumExtParam = 1;
        sts = m_pMfxENC->GetVideoParam(&encPar);    // read sps parameters from encoder
        assert(sts == MFX_ERR_NONE);

        sps_t sps;
        int ref_idc;
        h264bs_DecodeSps(extSPSPPS.SPSBuffer,extSPSPPS.SPSBufSize,sps,&ref_idc);    // parse parameters to sps_t struct
        // modify sps parameters
        {
            // read existing stream's sps parameters
            sps_t spsSource;
            BYTE * spsBuffer2 = new BYTE[1024];
            int spsSize2 = CFile(_T("c:\\ABRec\\ABNetTest.h264.sps"),CFile::modeRead).Read(spsBuffer2,1024);
            h264bs_DecodeSps(spsBuffer2,spsSize2,spsSource);
            delete []spsBuffer2;

            sps.log2_max_frame_num_minus4 = spsSource.log2_max_frame_num_minus4;

            // Not Support
            sps.seq_scaling_matrix_present_flag = 1;
            memcpy(sps.seq_scaling_list_present_flag,spsSource.seq_scaling_list_present_flag,sizeof(int)*8);
            memcpy(sps.ScalingList4x4,spsSource.ScalingList4x4,sizeof(int)*6*16);
            memcpy(sps.UseDefaultScalingMatrix4x4Flag,spsSource.UseDefaultScalingMatrix4x4Flag,sizeof(int)*6);
            memcpy(sps.ScalingList8x8,spsSource.ScalingList8x8,sizeof(int)*2*64);
            memcpy(sps.UseDefaultScalingMatrix8x8Flag,spsSource.UseDefaultScalingMatrix8x8Flag,sizeof(int)*2);

            // Not Support
            /sps.num_ref_frames = spsSource.num_ref_frames;

            sps.vui.aspect_ratio_info_present_flag = spsSource.vui.aspect_ratio_info_present_flag;

            assert(sps.vui.video_signal_type_present_flag != spsSource.vui.video_signal_type_present_flag);
            sps.vui.video_signal_type_present_flag = spsSource.vui.video_signal_type_present_flag;
            sps.vui.video_format = spsSource.vui.video_format;
            sps.vui.video_full_range_flag = spsSource.vui.video_full_range_flag;
            sps.vui.colour_description_present_flag = spsSource.vui.colour_description_present_flag;

            sps.vui.num_units_in_tick = spsSource.vui.num_units_in_tick;
            sps.vui.time_scale = spsSource.vui.time_scale;
            // Not Support
            sps.vui.fixed_frame_rate_flag = spsSource.vui.fixed_frame_rate_flag;

            // Not Support
            sps.hrd.bit_rate_value_minus1[0] = spsSource.hrd.bit_rate_value_minus1[0];
            sps.hrd.cpb_size_value_minus1[0] = spsSource.hrd.cpb_size_value_minus1[0];

            sps.vui.low_delay_hrd_flag = spsSource.vui.low_delay_hrd_flag;

            sps.vui.pic_struct_present_flag = spsSource.vui.pic_struct_present_flag;

            sps.vui.max_bytes_per_pic_denom = spsSource.vui.max_bytes_per_pic_denom;
            sps.vui.max_bits_per_mb_denom = spsSource.vui.max_bits_per_mb_denom;
            sps.vui.log2_max_mv_length_horizontal = spsSource.vui.log2_max_mv_length_horizontal;
            sps.vui.log2_max_mv_length_vertical = spsSource.vui.log2_max_mv_length_vertical;
            // Not Support
            sps.vui.max_dec_frame_buffering = spsSource.vui.max_dec_frame_buffering;
        }
        int spsSize = 0;
        h264bs_EncodeSps(sps,extSPSPPS.SPSBuffer,1024,spsSize);    // reencode sps to nalu rbsp format
        extSPSPPS.SPSBufSize = spsSize;

        sts = m_pMfxENC->Reset(&encPar);    // Reset Encoder,but return  MFX_ERR_INCOMPATIBLE_VIDEO_PARAM
        assert(sts == MFX_ERR_NONE);

        delete []spsBuffer;
        delete []ppsBuffer;
    }

it seems that some sps parameters can not be modified,or not supported by msdk encoder,but these parameters must be designated value,otherwise output stream will not be compatible with existing stream;so,what to do to make msdk encoder use this sps ?

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zheng,

This is not expected, so I can further look into the issue , can you send me your application code and bitstream (Source file) from which you are reading special SPS parameters to be set in msdk encoder. You can directly send me the package via"Send Author A Message".

Thanks,

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zheng,

After investing the issue I have following updates on the issue with settings up special SPS parameters. Currently the main reason the error message MFX_ERR_INCOMPATIBLE_VIDEO_PARAM is seen during query is due to scaling matrix. MSDK doesn't support this setting at the moment as no support at hw level is included.  

Thanks, 

0 Kudos
Zheng_L_
Beginner
953 Views

Harsh Jain (Intel) wrote:

Hi Zheng,

After investing the issue I have following updates on the issue with settings up special SPS parameters. Currently the main reason the error message MFX_ERR_INCOMPATIBLE_VIDEO_PARAM is seen during query is due to scaling matrix. MSDK doesn't support this setting at the moment as no support at hw level is included.  

Thanks, 

Thanks,Harsh! Did you mean that current version msdk encoder can't support this set of sps?

0 Kudos
Harshdeep_B_Intel
953 Views

Hi Zheng,

Yes, currently MSDK encoder does not support setting of Scaling matrix parameters. Please reply to private message and let me know if support is required and business requirements. 

Thanks,

0 Kudos
Reply