Trouble adding H.264 SEI message (unregistered data)

[karl_lillevold]

It seems when I try to add an SEI message of unregistred data before encoding in the sample QS encoder, I am getting an invalid bitstream. The Intel sample_decoder can decode it, but neither mp4box nor mp4creator can parse the bitstream ("Error decoding sei message")

Here is the snippet of code I am using. Could someone please advice what I am missing to get this to work. (I intend to put more/different info in CreateUserSEIPayload, this is just a sample)

[cpp]char * CEncodingPipeline::CreateUserSEIPayload()
{

	static const mfxU8 uuid[16] =
	{ 0xf1, 0xb6, 0x80, 0x14, 0x1c, 0xe2, 0x42, 0x4b, 
		0xaa, 0x6d, 0x4c, 0xbf, 0xee, 0x7c, 0x9c, 0x9b};
	// Generated with MSVC GUID Gen 06/22/2012

	int len = 1000;
	char *buffer, *s;
	buffer = s = (char *)malloc(len);
	if (!buffer)
		return NULL;


	memcpy(s,uuid,16);
	s += 16;

	mfxIMPL impl;
	m_mfxSession.QueryIMPL(&impl);
	char* sImpl = (MFX_IMPL_HARDWARE == impl) ? ("hw") : ("sw");
	s += sprintf_s(s, len-(s-buffer), " Media SDK impl: %s ", sImpl);

	return buffer;
}



memset(&encCtrl, 0, sizeof(encCtrl));
m_pszUserSEI = (mfxU8 *)CreateUserSEIPayload();

payload.Data = m_pszUserSEI;
payload.BufSize = (mfxU16)strlen((const char *)m_pszUserSEI)+1;
payload.NumBit = payload.BufSize * 8;
payload.Type = 5 ; // SEI USERDATA UNREGISTERED

encCtrl.NumPayload = 1;
parray = &payload;
encCtrl.Payload = &parray;
pEncCtrl = &encCtrl;

for (;;)
{
	// at this point surface for encoder contains either a frame 
	// from file or a frame processed by vpp        
	sts = m_pmfxENC->EncodeFrameAsync(pEncCtrl, &m_pEncSurfaces[nEncSurfIdx], 
		&pCurrentTask->mfxBS, &pCurrentTask->EncSyncP);
[/cpp]

[Petter_L_Intel]

Hi Karl,

It looks like you may be missing required SEI header info in your "m_pszUserSEI" data buffer. SEI message should start with SEI type and size of following buffer.

Take a look at chapter 4.13 in the Media SDK Developers Guide for details on how to populate SEI messages.

Hopefully this will help. Otherwise, please let us know.

Regards,

Petter

[karl_lillevold]

That sounds like what is needed, if only I could find that part of the documentation.. There are no chapters inmediasdk-man.pdf, and no chapter 4.13 inIntel_Media_Developers_Guide.pdf. However, in the mfxPayload description in mediasdk-man there's a reference to this and a pointer to the H.264 spec (which I had missed). I will try this approach.

Thanks!

[Petter_L_Intel]

Hmmm... sounds like you're stuck with old verison of the Media SDK. :)

If you download the latest release (2012 R2) of Media SDK from here, http://software.intel.com/en-us/articles/vcsource-tools-media-sdk/?cid=sem121p8191, you will find a version of the Media Developers Guide that has that chapter describing SEI message insertion.

Regards,

Petter

[karl_lillevold]

Indeed. I had 2012 R1.

Even with the two bytes added I was still getting an un-muxable ES, so then I copied and pasted the code in chapter 4.13 without any modifications, and added it to sample_encode in the Media SDK R2. Even then mp4creator (1.6.1d) reports the same "Error decoding sei message".. mp4box (via yamb) manages to create an MP4 file that most players can play, except Quicktime player (which also was the one with trouble before this correction).

[cpp]#define SEI_USER_DATA_REGISTERED_ITU_T_T35  4 
 
		typedef struct 
		{ 
			unsigned char countryCode; 
			unsigned char countryCodeExtension; 
			unsigned char payloadBytes[10];  // Containing arbitrary captions 
		} userdata_reg_t35; 

		// Insert SEI_USER_DATA_REGISTERED_ITU_T_T35 into payload 
		userdata_reg_t35 m_userSEIData; 
		m_userSEIData.countryCode           = 0xB5; 
		m_userSEIData.countryCodeExtension  = 0x31; 
		m_userSEIData.payloadBytes[0]       = 0x41; // payloadBytes[] containing captions 
 
		mfxU8 m_seiData[100]; // Arbitrary size 
		mfxPayload m_mySEIPayload; 
		MSDK_ZERO_MEMORY(m_mySEIPayload); 
		m_mySEIPayload.Type    = SEI_USER_DATA_REGISTERED_ITU_T_T35; 
		m_mySEIPayload.BufSize = sizeof(userdata_reg_t35) + 2;  // 2 bytes for header 
		m_mySEIPayload.NumBit  = m_mySEIPayload.BufSize * 8; 
		m_mySEIPayload.Data    = m_seiData; 
 
		// Insert SEI header and SEI msg into data buffer 
		m_seiData[0] = (mfxU8)m_mySEIPayload.Type;         // SEI type 
		m_seiData[1] = (mfxU8)(m_mySEIPayload.BufSize - 2); // Size of following msg 
		memcpy(m_seiData+2, &m_userSEIData, sizeof(userdata_reg_t35)); 
 
		mfxPayload* m_payloads[1];   
		m_payloads[0] = &m_mySEIPayload;  
 
		// Encode control structure initialization 
		MSDK_ZERO_MEMORY(m_encodeCtrl); 
		m_encodeCtrl.Payload = (mfxPayload**)&m_payloads[0]; 
		m_encodeCtrl.NumPayload = 1; 
		pEncCtrl = &m_encodeCtrl;

        for (;;)
        {
            // at this point surface for encoder contains either a frame from file or a frame processed by vpp        
            sts = m_pmfxENC->EncodeFrameAsync(pEncCtrl, &m_pEncSurfaces[nEncSurfIdx], &pCurrentTask->mfxBS, &pCurrentTask->EncSyncP);
[/cpp]

[karl_lillevold]

I got it to work now, at least playback in Quicktime player. My unregistered payload was more than 255 in length so I needed to follow the H.264 spec section 7.3.2.3.1. With regards to mp4creator and the sample code - maybe it does not handle all or any SEI messages?