Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3071 Discussions

H.264 Encoder inserts extra bytes before User Data SEI

Rotem
Beginner
1,059 Views

I am Using sample_encode for encoding H.264 and inserting "User Data Unregistered SEI".  

I am getting unexpected result:  

It looks like the encoder inserts few redundant (unexplained) extra bytes before the User Data.  

 

For testing, I created an input file in NV12 format using FFmpeg CLI:  

 

ffmpeg -f lavfi -i testsrc=size=320x256:rate=1:duration=50 -pix_fmt nv12 -f rawvideo in.nv12

 

I used the following command for encoding the raw video, and adding the User Data:  

 

sample_encode.exe h264 -nv12 -f 1 -n 50 -b 100 -g 10 -hw -async 1 -avbr -d3d11 -usei 4142434445464748494A4B4C4D4E4F50:0:MSDK -i in.nv12 -o out.264 -w 320 -h 256

 

The UUID is ABCDEFGHIJKLMNOP, and the data is MSDK.

 

When inspecting out.264, I see that the encoder inserts few extra bytes before the user data.

  • Inserts 6 extra bytes when encoding a key frame.
  • Inserts 3 extra bytes when encoding a P/B frame.

The issue is reproduced with Hardware encoder and Software encoder, and in Intel Core i7 Gen 6, and Core i5 Gen 3 CPUs, using Windows 10.   

Our customer is unable to decode the user data with the "extra bytes"...  

 

The first User Data NAL Unit (applies key frame) looks as follows:   

00 00 00 01 06 00 01 C0 01 01 04 05 14 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

 

The second User Data NAL Unit (applies non-key frame) looks as follows:  

00 00 00 01 06 01 01 04 05 14 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

 

00 00 00 01 - Start code prefix

06 - Header byte

6 or 3 bytes in bold are redundant (unexplained)

05 - NAL Unit type

14 - Data length

41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B - UUID and User Data

 

Using the debugger, I saw that the Payload excludes the extra bytes:   

 

In.pCtrl->Payload[0].Data = "x5x14ABCDEFGHIJKLMNOPMSDK<u+200e><u+200e><u+200e><u+200e>"
ayload[0].Data = "x5x14ABCDE

 nbsp;d[0].Data = "x5x14ABCDE

nbsp;d[0].Data = "x5x14ABCDE

The extra bytes are inserted by theLMNOPMSDK<u+200e><u+200e><u+ nbsp;tra bytes are inserted m_pmfxENC-bytes are inserted by theLMN>gt;fxENC-bytes are inserted EncodeFrameAsync, without user (programmer) control.   

 

Using FFmpeg as reference, the user data looks as it supposed to be:  

 

ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=1:duration=10 -vcodec libx264 -g 2 -pix_fmt yuv420p -bsf:v dump_extra=all,h264_metadata=sei_user_data='4142434445464748494A4B4C4D4E4F50+MSDK' vid.264

 

User data NAL Unit (FFmpeg result):  

00 00 01 06 05 15 41 42 43 44 45 46 47 48 49 4A 4B 4C 4D 4E 4F 50 4D 53 44 4B

 

Is it a bug, or am I missing something?  

 

Labels (2)
0 Kudos
3 Replies
Rahila_T_Intel
Employee
1,025 Views

Hi,


Thanks for posting in Intel communities.


Thanks for sharing all the details. We are working on this case internally. We will get back to u soon.



Thanks


0 Kudos
Rotem
Beginner
999 Views

I did some research, and realized that the inserted bytes matches the H.264 standard specification.  

 

According to the specification, few SEI "data units" are allowed to be concatenated - placed one after the other in the same NAL unit.

 

01 01 04 matches the following definitions:

  • 01 - Data type
  • 01 - Data length
  • 0C - Data (one byte)

01 01 04 - The next 3 bytes applies the next unit:

  • 01 - Data type
  • 01 - Data length
  • 04 - Data (one byte)

After the two "data units", comes our "data unit": 05 14 41 42 43 44 45 46 ...

05 - Data type

14 - Data length

Followed by 20 bytes of data.

 

I realized that the format matches the specification.  

Thank you

 

0 Kudos
Rahila_T_Intel
Employee
991 Views

Hi,


Glad to know that your issue is resolved. Thanks for sharing the solution with us.


If you need any additional information, please post a new question as this thread will no longer be monitored by Intel. 


Thanks


0 Kudos
Reply