Advanced MP4Splitter question

[moti_bot]

Hello,

I'm parsing an MP4 file using MP4Splitter class (part of UMC).
When using it with H264VideoDecoder, it works fine.
Also, parsing the MP4 file and reading frames as standalone works as well (correct number of frames are received) without decoding at all.

Now, when I take a look at the buffer I receive from GetNextData, it doesn't contain NAL information, but probably only frame data itself. The reason I ask that, is because I want to investigate another decoder using the same file parser, but it probably needs the NAL (with start codes).

When using the H264Splitter class, it behaves weird, the buffer from GetNextData includes NAL start codes, but it is incomplete, and it reads far more frames than exist in the file (and crashes very soon).

I woul appreciate if someone has some insight into that, or can advice on a method to build a NAL buffer from the output of the splitter (MP4 in our case or else).

Many thanks,
Moti.

[Chao_Y_Intel]

Hi Moti,

Do you mean SPS, PPS data are not returned by GetNextData() function in the splitter?
According to MP4 standard (14496 part 15?), there data are kept by the splitter internally.

You can check following data in UMC::SplitterInfo splitter to get these data:
m_ppTrackInfo[videoTrack]->m_pDecSpecInfo

Thanks,
Chao

[moti_bot]

Quoting - Chao Y (Intel)

Hi Moti,

Do you mean SPS, PPS data are not returned by GetNextData() function in the splitter?
According to MP4 standard (14496 part 15?), there data are kept by the splitter internally.

You can check following data in UMC::SplitterInfo splitter to get these data:
m_ppTrackInfo[videoTrack]->m_pDecSpecInfo

Thanks,
Chao

Hi Chao,

Thanks for your response.
Actually, what I'm looking for is to receive the full NAL packet, instead of only the NAL data itself.
Regarding SPS & PPS packet values (not just information) it would be also nice to get it in this way for another project I'm working on, but that's not bothering me at the moment.

I did start to use the MP4Splitter class for parsing MP4 files and passing then straight to the decoder, to get the MediaData with the final picture (in various color spaces).
In this model, there is a loop over the decoder, once a frame is received from the splitter in case it reports "not enough data", so another packet is read from the splitter (in both cases the return value is UMC_OK from the splitter).

I thought that would work with another decoder I'm working on now (not DirectDraw, pure software library), but it requires each frame with appropriate start code to be able to decode/parse it.
The H264Splitter does provide start codes (00 00 00 01 09...) for the data that is read, but it is not working so fluently, and not parsing the correct amount of frames from the track.

I would appreciate if there is a workaround or other solution for this.

Many thanks,
Moti.

[Victor_C_Intel1 · Accepted solution]

Hi!
The MP4Splitter and H264Splitter work different ways by intention.
When MP4Splitter was being developed the main goal was to be compatible with others MP4 splitters. They provide data without start codes. That is why it works like it works.

Data structure provided by the splitter has the following structure. Each frame may contains one or more NAL units. Every NAL unit is preceded by its size. The issue is that the size may have variable length - 1, 2 or 4 bytes. These information may be obtained from the DEC header(s).

So the instructions to convert mp4 data into pure h264 stream:
1. Read DEC header from the splitter
2. Parse length of 'size of NAL' variable
3. Change all 'size' variable into byte sequence 0 0 1. That's it.

Warning: Some streams may have length of 'size' variable equal to 1. So changing 'size' variable into thebyte sequence is impossible without additional copying.

PS: You can look at the end ofumc_h264_nal_spl.cpp file. The file contains some routines to parse DEC header and to change MP4 data into H264 pure data.

Best regards
Victor

[moti_bot]

Hi Victor,

Thanks, That is a very helpful reply, I would investigate into it further.
From your response I understand that the data I read from the splitter (MediaData) contains a size information at the beginning, followed by the data itself.
The number of bytes denoting the size can be read from the DEC header, so parsing the data would require to "create" a new buffer with start codes (00 00 00 01), then copying the actual data from the buffer.

Is the outline correct?

Regards,
Moti.

[Victor_C_Intel1]

Quoting - moti.bot

Hi Victor,

Thanks, That is a very helpful reply, I would investigate into it further.
From your response I understand that the data I read from the splitter (MediaData) contains a size information at the beginning, followed by the data itself.
The number of bytes denoting the size can be read from the DEC header, so parsing the data would require to "create" a new buffer with start codes (00 00 00 01), then copying the actual data from the buffer.

Is the outline correct?

Regards,
Moti.

Yes, you are correct

[moti_bot]

Quoting - Victor Cherepanov (Intel)

Yes, you are correct

Thank you, yet again!

:),
Moti.