MPEG-4 Video Decoder Problems

[franknatoli]

Have an MP4 file that Apple Quick Time has no problem displaying and that the IPP v5.2 library rejects with the first call to GetFrame. Cannot use IPP v5.3 library for reasons mentioned in an earlier post; MP4Splitter implementation is apparently broken for IPP v5.3. Have modified Intel Makefile for UMC libraries to use /Zi and /Od so debug information is generated and no optimization, so can debug the UMC code.

Am using the sample MPEG-4 decoder implementation in umc-manual.pdf, page 465, example 4-39, after correcting some mistakes, e.g., I use MP4Splitter not AVISplitter. Splitter.GetInfo gets good data, framerate=25.0, height=240, width=320, color_format=0, but curiously stream_type=0x180, which is H264_VIDEO not MPEG4_VIDEO (have tweaked example 4-39 to accept both H264_VIDEO and MPEG4_VIDEO).

GetFrame finds m_IsInit FALSE, hence calls InsideInit, which in turn calls mp4_SeekStartCodeOrShortPtr, which in turn calls mp4_FindStartCodeOrShortPtr.

mp4_FindStartCodeOrShortPtr looks for either start code prefix 00-00-01 or 00-00-80 which it fails to find within the 0x31 byte buffer that contains the MP4 data. The data in the MP4 buffer is clearly from the argument MP4 file. And there are 00-00-01 sequences in the MP4 file, but beyond the limits of the 0x31 byte buffer. mp4_FindStartCodeOrShortPtr fails to find the start code, and the error propagates backward, ultimately resulting in GetFrame returning -899, UM_ERR_INIT, indicating that the GetFrame init had failed.

Does IPP v5.2 work with MP4 files? I repeat, Apple Quick Time is perfectly happy with this file, and so is IPP v5.2, at least initially. But not for long.

Has anyone successfully decoded an MP4 file with IPP v5.2?

The subject MP4 file is 1.7Mb in case anyone want to see it.

Thanks for your time.

[franknatoli]

The answer is "no", simple_player does NOT successfully play the attached MP4 file. It does not generate any OUTPUT.YUV, although it does successfully decode the fundamental attributes of the file (as does my own program).

Should also note that I am using simple_player v5.2, because simple_player v5.3 has the MP4Splitter::GetInfo bug that I documented in a separate post (Intel issue number 466982) and doesn't even decode the fundamental attributes.

Have sent MP4 by direct e-mail.

Thanks for your time.

[franknatoli]

I knew it was H.264, but it never occurred to me that therefore the MPEG4VideoDecoder was the wrong choice. Thanks, will try your suggestion immediately.

[franknatoli]

Many thanks to Nikolay and f_c for their help.

My application is now nicely decoding both MPEG-4 and H.264. The "only" remaining problem is that the decoder appears to get stuck returning -996 UMC_ERR_NOT_ENOUGH_DATA, both for an MPEG-4 MP4 file and for an H.264 MP4 file, both of which play nicely with Quick Time v7.4. I've inserted some code, see below, to watch for the decoder getting stuck with UMC_ERR_NOT_ENOUGH_DATA and aborting in that event. For both files, the point where the decoder gets stuck with UMC_ERR_NOT_ENOUGH_DATA is apparently at end of file. Is it possible that the splitter has stopped running but failed to advise the decoder?


// AtsPlaybackMpeg4Proc.cpp
//

#include "stdafx.h"
#include "AtsPlaybackMpeg4.h"

#include "AtsPlaybackMpeg4Doc.h"
#include "AtsPlaybackMpeg4View.h"

#include "ipp.h"
#include "umc_file_reader.h"
#include "umc_fio_reader.h"
#include "umc_h264_dec.h"
#include "umc_h264_spl.h"
#include "umc_h264_timing.h"
#include "umc_mp4_spl.h"
#include "umc_mpeg4_video_decoder.h"
#include "umc_splitter.h"
#include "umc_video_render.h"
#include "fw_video_render.h"
#include "vm_time.h"

#ifdef _DEBUG
#define new DEBUG_NEW
#endif

void CAtsPlaybackMpeg4View::PlaybackThread(void)
{
CString str;
UMC::Status umcResult;

//-------------------------------------------------------------------------
// init file reader
//-------------------------------------------------------------------------
UMC::FIOReader reader;
UMC::FileReaderParams readerParams;
readerParams.m_portion_size = 0;
vm_string_strcpy(readerParams.m_file_name, (LPCTSTR)m_strMpeg4File);
umcResult = reader.Init(&readerParams);
if (umcResult != UMC::UMC_OK)
{
str.Format("FIOReader.Init failure %d", umcResult);
AfxMessageBox(str);
return;
}

//-------------------------------------------------------------------------
// init splitter
//-------------------------------------------------------------------------
UMC::MP4Splitter splitter;
UMC::SplitterParams splitterParams;
splitterParams.m_lFlags = UMC::VIDEO_SPLITTER;
splitterParams.m_pDataReader = &reader;
umcResult = splitter.Init(splitterParams);
if (umcResult != UMC::UMC_OK)
{
str.Format("Splitter.Init failure %d", umcResult);
AfxMessageBox(str);
return;
}

umcResult = splitter.Run();
if (umcResult != UMC::UMC_OK)
{
str.Format("Splitter.Run failure %d", umcResult);
AfxMessageBox(str);
return;
}

UMC::Spl itterInfo *streamInfo;
umcResult = splitter.GetInfo(&streamInfo);
if (umcResult != UMC::UMC_OK)
{
str.Format("Splitter.GetInfo failure %d", umcResult);
AfxMessageBox(str);
return;
}

Ipp32u videoTrack;
for (videoTrack = 0; videoTrack < streamInfo->m_nOfTracks; videoTrack++)
{
TRACE("videoTrack %d m_type 0x%X ", videoTrack, streamInfo->m_ppTrackInfo[videoTrack]->m_Type);
if (streamInfo->m_ppTrackInfo[videoTrack]->m_Type == UMC::TRACK_MPEG4V ||
streamInfo->m_ppTrackInfo[videoTrack]->m_Type == UMC::TRACK_H264)
break;
}

if (videoTrack == streamInfo->m_nOfTracks)
{
AfxMessageBox("MP4 does not contain TRACK_MPEG4V or TRACK_H264");
return;
}

UMC::VideoStreamInfo &videoTrackInfo = *((UMC::VideoStreamInfo *)streamInfo->m_ppTrackInfo[videoTrack]->m_pStreamInfo);

TRACE("m_nOfTracks %d videoTrack %d stream_type 0x%X framerate %.1f clip_info.width %d clip_info.height %d color_format %d ",
streamInfo->m_nOfTracks,
videoTrack,
videoTrackInfo.stream_type,
videoTrackInfo.framerate,
videoTrackInfo.clip_info.width,
videoTrackInfo.clip_info.height,
videoTrackInfo.color_format);

//-------------------------------------------------------------------------
// init video decoder
//-------------------------------------------------------------------------
UMC::VideoDecoder *decoder;
switch (videoTrackInfo.stream_type)
{
case UMC::MPEG4_VIDEO :
decoder = (UMC::VideoDecoder*)(new UMC::MPEG4VideoDecoder());
break;
case UMC::H264_VIDEO :
decoder = (UMC::VideoDecoder*)(new UMC::H264VideoDecoder());
break;
default :
str.Format("stream_type %d neither MPEG4_VIDEO nor H264_VIDEO", videoTrackInfo.stream_type);
AfxMessageBox(str);
return;
}

UMC::VideoDecoderParams decoderParams;
decoderParams.info = videoTrackInfo;
decoderParams.numThreads = 1;
decoderParams.lFlags = UMC::FLAG_VDEC_REORDER;
decoderParams.m_pData = stream Info->m_ppTrackInfo[videoTrack]->m_pDecSpecInfo;
umcResult = decoder->Init(&decoderParams);
if (umcResult != UMC::UMC_OK)
{
str.Format("MPEG4VideoDecoder.Init failure %d", umcResult);
AfxMessageBox(str);
return;
}

//-------------------------------------------------------------------------
// init video renderer
//-------------------------------------------------------------------------
UMC::VideoRenderParams renderParams;
umcResult = renderParams.out_data_template.Init(
videoTrackInfo.clip_info.width,
videoTrackInfo.clip_info.height,
videoTrackInfo.color_format);
if (umcResult != UMC::UMC_OK)
{
str.Format("VideoRenderParams.Init failure %d", umcResult);
AfxMessageBox(str);
return;
}

UMC::FWVideoRender render;
umcResult = render.Init(&renderParams);
if (umcResult != UMC::UMC_OK)
{
str.Format("FWVideoRender.Init failure %d", umcResult);
AfxMessageBox(str);
return;
}

//-------------------------------------------------------------------------
// process frames
//-------------------------------------------------------------------------
UMC::MediaData dataIn;
UMC::VideoData dataOut;
vm_tick total = 0;
vm_tick freq = vm_time_get_frequency();
int nframes = 0;
int decoderWait = 0;
while (1)
{
while (1)
{
umcResult = splitter.GetNextData(&dataIn, videoTrack);
if (umcResult != UMC::UMC_ERR_NOT_ENOUGH_DATA)
break;
vm_time_sleep(5);
}

if (umcResult != UMC::UMC_OK &&
umcResult != UMC::UMC_ERR_END_OF_STREAM)
{
str.Format("Splitter.GetNextData failure %d", umcResult);
AfxMessageBox(str);
break;
}

render.LockInputBuffer(&dataOut);
vm_tick t0 = vm_time_get_tick();

// if call to GetNextVideoData was entirely successful then pass MediaData input to GetFrame
if (umcResult == UMC::UMC_OK)
umcResult = decoder->GetFrame(&dataIn, &dataOut);
// if call to GetNextVideoData was not entirely successful then pass NULL input to GetFrame
else
umcResult = decoder->GetFrame(NULL, &dataOut);

vm_tick t1 = vm_time_get_tick();
total += t1 - t0;

// if call to GetFrame resulted in end of stream
if (umcResult == UMC::UMC_ERR_END_OF_STREAM)
{
AfxMessageBox("decoder.GetFrame returned UMC_ERR_END_OF_STREAM");
break;
}

// if call to GetFrame resulted in fatal error
if (umcResult != UMC::UMC_OK &&
umcResult != UMC::UMC_ERR_NOT_ENOUGH_DATA)
{
str.Format("decoder.GetFrame failure %d", umcResult);
AfxMessageBox(str);
break;
}

render.UnLockInputBuffer(&dataOut);

// if call to GetFrame was entirely successful then render output frame
if (umcResult == UMC::UMC_OK)
{
TRACE("frame %d delta %d FPS %.1f ", nframes, t1 - t0, 1.0 / ((t1 - t0) / (double)freq));
Ipp64f time = -1;
while (render.GetRenderFrame(&time) == UMC::UMC_ERR_TIMEOUT)
AfxMessageBox("render.GetRenderFrame UMC_ERR_TIMEOUT");
umcResult = render.RenderFrame();
if (umcResult != UMC::UMC_OK)
{
str.Format("render.RenderFrame failure %d", umcResult);
AfxMessageBox(str);
break;
& nbsp;}
nframes++;
decoderWait = 0;
}
// check for stuck at UMC_ERR_NOT_ENOUGH_DATA
else if (++decoderWait >= 100)
{
AfxMessageBox("decoder stuck at UMC_ERR_NOT_ENOUGH_DATA");
return;
}
}
TRACE("exiting ");
}

DWORD WINAPI CAtsPlaybackMpeg4View::StartPlaybackThread(LPVOID arg)
{
CAtsPlaybackMpeg4View *lpView = (CAtsPlaybackMpeg4View*)arg;
lpView->PlaybackThread();
return(0);
}

[frankjnatoli]

Yes, but, the evidence is that sometimes the splitter fails to return UMC_ERR_END_OF_STREAM and rather simply returns, endlessly, UMC_ERR_NOT_ENOUGH_DATA. And if the splitter does not return UMC_OK, the renderer should not be called, hence cannot be tested for a status of its own. Ergo, it would appear necessary to diagnose the splitter getting stuck with UMC_ERR_NOT_ENOUGH_DATA.

[fido_genial]

Hi FrankNatol,

Thanks for your suggestion. Please can you tell me will this code work for a Mpeg4 video stream with No audio i.e, my video file starts with an MPEG4 Visual Object Sequence header (0x000001b0). Will MP4splitter can consider this video has MPEG-4 Pure Video Stream and tracks = 0 in splitter code.
please can you suggest how to decode this file .

Thanks
fido

[franknatoli]

The code that I posted only initializes the splitter for video data, so the code does not care whether or not there is any audio data in the stream. The splitter will discard the audio if present.

If your video is only a few megabytes, you can send it to me at frankjnatoli-at-embarqmail.com. I'll run it through the above application.

The UMC::SplitterParams field m_lFlags can be programmed with UMC::VIDEO_SPLITTER, as I do above, which simply extracts video and discards audio, or can be programmed with UMC::AUDIO_SPLITTER, which simply extracts audio and discards video, or can be programmed with an inclusive-OR of both, which extracts both video and audio. It depends on how you implement your threads. One thread for video and a separate thread for audio should program the splitter separately. A single thread for video and audio should program the splitter with the inclusive-OR of the flags.