Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3043 Discussions

Trouble adding H.264 SEI message (unregistered data)

karl_lillevold
Beginner
1,096 Views
It seems when I try to add an SEI message of unregistred data before encoding in the sample QS encoder, I am getting an invalid bitstream. The Intel sample_decoder can decode it, but neither mp4box nor mp4creator can parse the bitstream ("Error decoding sei message")

Here is the snippet of code I am using. Could someone please advice what I am missing to get this to work. (I intend to put more/different info in CreateUserSEIPayload, this is just a sample)

[cpp]char * CEncodingPipeline::CreateUserSEIPayload()
{

	static const mfxU8 uuid[16] =
	{ 0xf1, 0xb6, 0x80, 0x14, 0x1c, 0xe2, 0x42, 0x4b, 
		0xaa, 0x6d, 0x4c, 0xbf, 0xee, 0x7c, 0x9c, 0x9b};
	// Generated with MSVC GUID Gen 06/22/2012

	int len = 1000;
	char *buffer, *s;
	buffer = s = (char *)malloc(len);
	if (!buffer)
		return NULL;


	memcpy(s,uuid,16);
	s += 16;

	mfxIMPL impl;
	m_mfxSession.QueryIMPL(&impl);
	char* sImpl = (MFX_IMPL_HARDWARE == impl) ? ("hw") : ("sw");
	s += sprintf_s(s, len-(s-buffer), " Media SDK impl: %s ", sImpl);

	return buffer;
}



memset(&encCtrl, 0, sizeof(encCtrl));
m_pszUserSEI = (mfxU8 *)CreateUserSEIPayload();

payload.Data = m_pszUserSEI;
payload.BufSize = (mfxU16)strlen((const char *)m_pszUserSEI)+1;
payload.NumBit = payload.BufSize * 8;
payload.Type = 5 ; // SEI USERDATA UNREGISTERED

encCtrl.NumPayload = 1;
parray = &payload;
encCtrl.Payload = &parray;
pEncCtrl = &encCtrl;

for (;;)
{
	// at this point surface for encoder contains either a frame 
	// from file or a frame processed by vpp        
	sts = m_pmfxENC->EncodeFrameAsync(pEncCtrl, &m_pEncSurfaces[nEncSurfIdx], 
		&pCurrentTask->mfxBS, &pCurrentTask->EncSyncP);
[/cpp]

0 Kudos
5 Replies
Petter_L_Intel
Employee
1,096 Views
Hi Karl,

It looks like you may be missing required SEI header info in your "m_pszUserSEI" data buffer. SEI message should start with SEI type and size of following buffer.

Take a look at chapter 4.13 in the Media SDK Developers Guide for details on how to populate SEI messages.

Hopefully this will help. Otherwise, please let us know.

Regards,
Petter
0 Kudos
karl_lillevold
Beginner
1,096 Views
That sounds like what is needed, if only I could find that part of the documentation.. There are no chapters inmediasdk-man.pdf, and no chapter 4.13 inIntel_Media_Developers_Guide.pdf. However, in the mfxPayload description in mediasdk-man there's a reference to this and a pointer to the H.264 spec (which I had missed). I will try this approach.

Thanks!

0 Kudos
Petter_L_Intel
Employee
1,096 Views
Hmmm... sounds like you're stuck with old verison of the Media SDK. :)

If you download the latest release (2012 R2) of Media SDK from here, http://software.intel.com/en-us/articles/vcsource-tools-media-sdk/?cid=sem121p8191, you will find a version of the Media Developers Guide that has that chapter describing SEI message insertion.

Regards,
Petter
0 Kudos
karl_lillevold
Beginner
1,096 Views
Indeed. I had 2012 R1.
Even with the two bytes added I was still getting an un-muxable ES, so then I copied and pasted the code in chapter 4.13 without any modifications, and added it to sample_encode in the Media SDK R2. Even then mp4creator (1.6.1d) reports the same "Error decoding sei message".. mp4box (via yamb) manages to create an MP4 file that most players can play, except Quicktime player (which also was the one with trouble before this correction).

[cpp]#define SEI_USER_DATA_REGISTERED_ITU_T_T35  4 
 
		typedef struct 
		{ 
			unsigned char countryCode; 
			unsigned char countryCodeExtension; 
			unsigned char payloadBytes[10];  // Containing arbitrary captions 
		} userdata_reg_t35; 

		// Insert SEI_USER_DATA_REGISTERED_ITU_T_T35 into payload 
		userdata_reg_t35 m_userSEIData; 
		m_userSEIData.countryCode           = 0xB5; 
		m_userSEIData.countryCodeExtension  = 0x31; 
		m_userSEIData.payloadBytes[0]       = 0x41; // payloadBytes[] containing captions 
 
		mfxU8 m_seiData[100]; // Arbitrary size 
		mfxPayload m_mySEIPayload; 
		MSDK_ZERO_MEMORY(m_mySEIPayload); 
		m_mySEIPayload.Type    = SEI_USER_DATA_REGISTERED_ITU_T_T35; 
		m_mySEIPayload.BufSize = sizeof(userdata_reg_t35) + 2;  // 2 bytes for header 
		m_mySEIPayload.NumBit  = m_mySEIPayload.BufSize * 8; 
		m_mySEIPayload.Data    = m_seiData; 
 
		// Insert SEI header and SEI msg into data buffer 
		m_seiData[0] = (mfxU8)m_mySEIPayload.Type;         // SEI type 
		m_seiData[1] = (mfxU8)(m_mySEIPayload.BufSize - 2); // Size of following msg 
		memcpy(m_seiData+2, &m_userSEIData, sizeof(userdata_reg_t35)); 
 
		mfxPayload* m_payloads[1];   
		m_payloads[0] = &m_mySEIPayload;  
 
		// Encode control structure initialization 
		MSDK_ZERO_MEMORY(m_encodeCtrl); 
		m_encodeCtrl.Payload = (mfxPayload**)&m_payloads[0]; 
		m_encodeCtrl.NumPayload = 1; 
		pEncCtrl = &m_encodeCtrl;

        for (;;)
        {
            // at this point surface for encoder contains either a frame from file or a frame processed by vpp        
            sts = m_pmfxENC->EncodeFrameAsync(pEncCtrl, &m_pEncSurfaces[nEncSurfIdx], &pCurrentTask->mfxBS, &pCurrentTask->EncSyncP);
[/cpp]

0 Kudos
karl_lillevold
Beginner
1,096 Views
I got it to work now, at least playback in Quicktime player. My unregistered payload was more than 255 in length so I needed to follow the H.264 spec section 7.3.2.3.1. With regards to mp4creator and the sample code - maybe it does not handle all or any SEI messages?
0 Kudos
Reply