MP4 muxer with H264 encoder - debug vs release

Madhan_S — Wed, 30 Jul 2014 08:03:26 GMT

Hi,

I have some grayscale frames and I need to create an mp4 video with H264 encoding out of it. I am using Intel IPP Samples 7.0.1.

The application works fine in debug but doesn't in release configuration. The mp4 output in release has the correct size, duration and dimensions, but the image wont play in any player.

This is not the case in debug mode or when I do debug with breakpoints in release mode (which I guess makes the execution a bit slow for the encoder/muxer to work properly).After some investigation, it seems that due to optimizations in release mode, the execution is very fast and hence proper mp4 output is not generated.

he documentation says that according to the H264 algorithm there could be some B frames pending in the buffer and they need flushed, so I tried passing NULL to the encoder and retrieve pending frames. But even that didnt help in release mode.

Should I try flushing the muxer? but at what stage?

I am attaching the input 8 bit grayscale image (1024 X 1024 X 100 frames).

Please find my code below

//initialize code for the mp4 creator
bool MP4::Initialize(const int& numFrames, const int& frameWidth, const int& frameHeight, const int& frameRate, const std::string& fullfilepath)
{
	m_width = frameWidth;
	m_height = frameHeight;
	m_bitDepth = 8;
	m_numFrames = numFrames;
	m_currentFrameIndex = 0;
	

	vm_string_strcpy(writerParams.m_file_name, fullfilepath.c_str());
	writerParams.m_portion_size = 0;
	result = writer.Init(&writerParams);

	videoInfo.clip_info.height = frameHeight;
	videoInfo.clip_info.width = frameWidth;
	videoInfo.stream_type = UMC::VideoStreamType::H264_VIDEO;
	videoInfo.color_format = UMC::ColorFormat::YUV420;
	videoInfo.interlace_type = UMC::InterlaceType::PROGRESSIVE;
	videoInfo.bitrate = 10000000;
	videoInfo.framerate = float(frameRate);
	videoInfo.streamPID = 0;
	videoInfo.duration = float(numFrames) / frameRate;

	muxerParams.m_lpDataWriter = &writer;
	muxerParams.m_SystemType = UMC::SystemStreamType::H264_PURE_VIDEO_STREAM;
	muxerParams.m_lFlags = FLAG_FRAGMENTED_AT_I_PICTURES;
	muxerParams.m_nNumberOfTracks = 1;
	muxerParams.pTrackParams = new UMC::TrackParams[1];//&videoTrackParams;
	muxerParams.pTrackParams[0].type = UMC::VIDEO_TRACK;
	muxerParams.pTrackParams[0].info.video = &videoInfo;
	muxerParams.pTrackParams[0].bufferParams.m_numberOfFrames = numFrames;
	

	result = m_MP4muxer.Init(&muxerParams);
	
	m_H264EncoderParams.key_frame_controls.method=1;
	m_H264EncoderParams.info.clip_info.height=frameHeight;
	m_H264EncoderParams.info.clip_info.width=frameWidth;
	m_H264EncoderParams.info.bitrate = 10000000;
	m_H264EncoderParams.numThreads = 1;
	m_H264EncoderParams.chroma_format_idc = 1;
	m_H264EncoderParams.numFramesToEncode = numFrames;
	m_H264EncoderParams.info.duration = (float)numFrames/frameRate;
	m_H264EncoderParams.info.framerate = frameRate;
	
	result = m_H264Encoder.Init(&m_H264EncoderParams);
	return result == UMC_OK;
}

//Add frame code for mp4 creator
bool MP4::AddFrame(void* frameData) // framedata is 8 bit grayscale image
{
	int yuvframeSize = m_width * m_height * 3/2;
	Ipp8u* yuvPixels = ippsMalloc_8u(yuvframeSize);
	memset(yuvPixels, 127, yuvframeSize); 						//neutral values for u and v components
	memcpy(yuvPixels, frameData, m_width * m_height); // grayscale to yuv completed

	UMC::VideoData uncompressedData;
	uncompressedData.Init(m_width, m_height, UMC::ColorFormat::YUV420, m_bitDepth);
	uncompressedData.SetBufferPointer(yuvPixels, yuvframeSize);
	uncompressedData.SetDataSize(yuvframeSize);

	UMC::MediaData videoData;
	result = m_MP4muxer.LockBuffer(&videoData, 0);
	result = m_H264Encoder.GetFrame(&uncompressedData, &videoData);

	Ipp64f start = (m_currentFrameIndex/(float)m_numFrames)*m_H264EncoderParams.info.duration;
	Ipp64f end = ((m_currentFrameIndex+1)/(float)m_numFrames)*m_H264EncoderParams.info.duration;
	videoData.SetTime(start,end);
	
	result = m_MP4muxer.UnlockBuffer(&videoData,0);

	ippsFree(yuvPixels);
	m_currentFrameIndex++;
	return result == UMC::UMC_OK;
}

bool MP4::::Close()
{
	UMC::Status status = UMC::UMC_OK;
	status = m_MP4muxer.PutEndOfStream(0);
	status = m_MP4muxer.Close();
	delete [] muxerParams.pTrackParams;
	return status == UMC::UMC_OK;
}

Client Code

#include <fstream>
#include <mp4.h>
using namespace std;
using namespace IntelIPP;

void main()
{
	int width = 1024;
	int height = 1024;
	int numFrames = 100;
	int frameRate = 10;
	int frameSize = width*height;

	char* rawData = new char[frameSize* numFrames];

	ifstream fxdFile;
	fxdFile.open("D:\\Temp\\testImage.raw",std::ios::binary);
	fxdFile.read(rawData, frameSize*numFrames);
	fxdFile.close();

	MP4 mp4creator;
	mp4creator.Initialize(numFrames, width, height, frameRate, "D:\\Temp\\video.mp4");

	char* frame = new char[frameSize];
	for(int i = 0; i<numFrames; i++)
	{
		memcpy(frame, rawData + i*frameSize, frameSize);
		mp4creator.AddFrame(frame);
	}

	mp4creator.Close();
}

I am attaching the mp4 videos

Madhan_S — Wed, 30 Jul 2014 08:13:40 GMT

I am attaching the mp4 videos generated in release and debug mode. And the hex comparison between the two. Please note that mostly the frame data appears to be the same, and there is a pattern in the differences - 1 bit lesser in the non working video

Could not find the actual

Madhan_S — Wed, 30 Jul 2014 12:54:45 GMT

Could not find the actual reason but the stco table values were all offset by 16.

muxerParams.m_lFlags = FLAG_FRAGMENTED_AT_I_PICTURES;

I removed explicit setting of the muxerParams flags as shown above and went along with the default initialization. And it worked !

Hi Madhan S,

Ying_H_Intel — Thu, 31 Jul 2014 02:09:38 GMT

Hi Madhan S,

Glad to know it works finally. We will investigate the reason if possible.

Thanks

Ying

Hi Madhan,Thank you very much

Ying_H_Intel — Wed, 20 Aug 2014 07:59:58 GMT

Hi Madhan,

Thank you very much for the sharing. We confirm there are several dependent bugs with the Flags, which cause the header offset size error in some condition. and right, the solution is to remove the flags setting.

Best Regards,

Ying

Re: Hi Madhan,Thank you very much

AF5 — Tue, 01 Dec 2020 07:58:39 GMT

Hello,

I know it's 6 years late, but still,

were any of the issues with FLAG_FRAGMENTED_AT_I_PICTURES addressed?

If yes, is there any chance to get a diff with 7.1.1.013 samples or just the code for mp4muxer?

Thanks.

Re: Hi Madhan,Thank you very much

Gennady_F_Intel — Wed, 02 Dec 2020 07:54:10 GMT

Hello, this issue has not been addressed and I have to say that this functionality is not part of the current version of IPP anymore.

topic Hi Madhan,Thank you very much in Intel® Integrated Performance Primitives

MP4 muxer with H264 encoder - debug vs release

I am attaching the mp4 videos

Could not find the actual

Hi Madhan S,

Hi Madhan,Thank you very much

Re: Hi Madhan,Thank you very much

Re: Hi Madhan,Thank you very much