H.264 Encoder inserts extra bytes before User Data SEI

[Rotem]

I am Using sample_encode for encoding H.264 and inserting "User Data Unregistered SEI".  

I am getting unexpected result:  

It looks like the encoder inserts few redundant (unexplained) extra bytes before the User Data.  

For testing, I created an input file in NV12 format using FFmpeg CLI:  

ffmpeg -f lavfi -i testsrc=size=320x256:rate=1:duration=50 -pix_fmt nv12 -f rawvideo in.nv12

I used the following command for encoding the raw video, and adding the User Data:  

sample_encode.exe h264 -nv12 -f 1 -n 50 -b 100 -g 10 -hw -async 1 -avbr -d3d11 -usei 4142434445464748494A4B4C4D4E4F50:0:MSDK -i in.nv12 -o out.264 -w 320 -h 256

The UUID is ABCDEFGHIJKLMNOP, and the data is MSDK.

When inspecting out.264, I see that the encoder inserts few extra bytes before the user data.

• Inserts 6 extra bytes when encoding a key frame.
• Inserts 3 extra bytes when encoding a P/B frame.

The issue is reproduced with Hardware encoder and Software encoder, and in Intel Core i7 Gen 6, and Core i5 Gen 3 CPUs, using Windows 10.   

Our customer is unable to decode the user data with the "extra bytes"...  

The first User Data NAL Unit (applies key frame) looks as follows:   

00 00 00 01 06 00 01 C0 01 01 04 05 14 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

The second User Data NAL Unit (applies non-key frame) looks as follows:  

00 00 00 01 06 01 01 04 05 14 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

00 00 00 01 - Start code prefix

06 - Header byte

6 or 3 bytes in bold are redundant (unexplained)

05 - NAL Unit type

14 - Data length

41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B - UUID and User Data

Using the debugger, I saw that the Payload excludes the extra bytes:   

In.pCtrl->Payload[0].Data = "x5x14ABCDEFGHIJKLMNOPMSDK<u+200e><u+200e><u+200e><u+200e>"

ayload[0].Data = "x5x14ABCDE

 nbsp;d[0].Data = "x5x14ABCDE

nbsp;d[0].Data = "x5x14ABCDE

The extra bytes are inserted by theLMNOPMSDK<u+200e><u+200e><u+ nbsp;tra bytes are inserted m_pmfxENC-bytes are inserted by theLMN>gt;fxENC-bytes are inserted EncodeFrameAsync, without user (programmer) control.   

Using FFmpeg as reference, the user data looks as it supposed to be:  

ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=1:duration=10 -vcodec libx264 -g 2 -pix_fmt yuv420p -bsf:v dump_extra=all,h264_metadata=sei_user_data='4142434445464748494A4B4C4D4E4F50+MSDK' vid.264

User data NAL Unit (FFmpeg result):  

00 00 01 06 05 15 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

Is it a bug, or am I missing something?  

[Rahila_T_Intel]

Hi,

Thanks for posting in Intel communities.

Thanks for sharing all the details. We are working on this case internally. We will get back to u soon.

Thanks

[Rotem]

I did some research, and realized that the inserted bytes matches the H.264 standard specification.  

According to the specification, few SEI "data units" are allowed to be concatenated - placed one after the other in the same NAL unit.

01 01 04 matches the following definitions:

• 01 - Data type
• 01 - Data length
• 0C - Data (one byte)

01 01 04 - The next 3 bytes applies the next unit:

• 01 - Data type
• 01 - Data length
• 04 - Data (one byte)

After the two "data units", comes our "data unit": 05 14 41 42 43 44 45 46 ...

05 - Data type

14 - Data length

Followed by 20 bytes of data.

I realized that the format matches the specification.  

Thank you

[Rahila_T_Intel]

Hi,

Glad to know that your issue is resolved. Thanks for sharing the solution with us.

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel. 

Thanks