Using MPEG-4 Decoder

[sdhays]

I'm trying to build my own application based on the UMC framework from the IPP Media Samples, v5.0.017. I can play AVI and MP4 files using simple_player, and MJPEG AVIs play fine with the UMC example (from the UMC manual). But when I change the UMC example to use MPEG4VideoDecoder instead of MJPEGVideoDecoder, the program runs, seeming to process every frame, however not one frame is decoded for rendering. I'm not surprised that the MPEG4 decoder needs to be treated differently from the MJPEG decoder, but after searching for the last 3 days through simple_player.cpp, avsync.cpp, codec_pipeline.cpp, and umc_mpeg4_video_decoder.cpp, I'm still at a loss as to what I should do to get the MPEG4 decoder to work.

Any suggestions would be greatly appreciated!
Scott


P.S. As further note for when Intel updates the documentation: FIOReader is not built by default on Linux for IA32. The UMC example depends on FIOReader and thus cannot be built without modification. Changing FIOReader to FileReader works, but I don't know what difference (if any) this makes.

Message Edited by sdhays@neon.com.tw on 01-19-2006 06:35 PM

[Vladimir_Dudnik]

Ops, sorry, I was able to reproduce issue (just mistakenly used latest sources).

Well, there is workaround for that issue, you need to call VideoData.SetVideoParameters() function before VideoRender.RenderFrame(). I've attached MSVC project for your convenience, please let me know if it work for you.

Note, you need to unzip umc_mpeg4_dec folder to UMCapplications folder. And you need to pre-build UMC static libraries as this project depends from prebuild libraries for similicity.

Regards,
Vladimir

[walkdeath]

hi~~~Thanks for your sample.
I've tried in linux(Fedora Core 4), using build32.sh to build the sample. But a "segment fault" problem came...
I found that the problem is in
if(umcRes == UMC::UMC_OK)

{

out.SetVideoParameters(

spl_info.m_video_info.clip_info.width,

spl_info.m_video_info.clip_info.height,

UMC::YV12);



umcRes = dst.UnLockInputBuffer(&out);

if (umcRes != UMC::UMC_OK)

break;



double time = dst.GetRenderFrame();



umcRes = dst.RenderFrame();

if (umcRes != UMC::UMC_OK)

break;

}
When I made "umcRes = dst.RenderFrame" a comment with "//".
Then there was no "segment fault" appeared, but it could only process 2 frames and also left me an empty "output.yuv" file.
And when I mad " out.SetVideoParameters..." a comment, it could process 11 frames and still, an empty "output.yuv" file.
Is there something wrong with the avi file that I process?
My avi file is 10M, so I can't attach it easily. Could anybody attach a small avi file?
Any suggestion will be appreciated!

[walkdeath]

I've just done some tested and found something intereting.
In function "DecodeMPEG4Video", I add some code to test if the media data is correct.

// decoding & rendering loop (frame by frame)

while(umcRes == UMC::UMC_OK || umcRes == UMC::UMC_NOT_ENOUGH_DATA)

{

umcRes = avi_spl.GetNextVideoData(&in);


if (umcRes != UMC::UMC_OK)

break;

// printf("%d ", in.GetDataSize());
//this test prove that there is data in "in" whose type is MediaData ;

but after the follows,
umcRes = mpeg4_dec.GetFrame(&in, &out);
printf("in:%d ", in.GetDataSize());
printf("out:%d ", out.GetDataSize());

if (umcRes != UMC::UMC_OK && umcRes != UMC::UMC_NOT_ENOUGH_DATA)

break;

I only get "in:1 out:0". I think this proves that there is something wrong with GetFrame. But strangely, the "umcRes" passed the "if" test!
That really made me confused...

[sdhays]

Thanks, Vladimir. Adding an out.SetVideoParameters() call fixed it for me on both Windows and Linux.

On a different, but related, topic: do you have a suggestion for how to play the output.yuv file? Can simpleplayer play it? If so, I haven't been able to get it to accept the output.yuv file. I've been playing it with YUV Viewer, a DirectShow plugin, but I was wondering if there was a better way. I haven't yet found the right command-line incantation to get MPlayer/FFmpeg to play it.

Thanks again for your help.
Scott

[walkdeath]

Hi, scott

I've tried Vladimir's code in Fedora Core 4, but problems just come as I said.

Didn't you meet any problems when run the decoder in linux? There is always a "segment fault" when I run it... Could you offer some suggestion?

Thanks,

Wu Yu

[sdhays]

Hi Wu Yu.
What file reader are you using? I had to change it to use the regular UMC::FileReader class for Linux. FIOReader doesn't even get built for Linux. I posted this issue earlier in another thread. Hopefully the next sample release will have a UMC sample that can work without any changes for both Windows and Linux.

#ifdef WIN32
UMC::FIOReader src;
#else
UMC::FileReader src;
#endif

That works for me on Red Hat 9.

Scott

[walkdeath]

Hi Scott,
I've changed the file reader to FileReader, but still the same problem, "segment fault". There is nothing in the "VideoData out".
Could you please attach your sample code? Thanks very much!

Wu Yu

[sdhays]

This is my edit of the file Vladimir originally posted. I've only added the setvideoparam() call that Vladimir added in his update (although my version only calls it once). This works for me on Red Hat 9.

Scott

[walkdeath]

Hi Scott,
I've tried your code , but still the same problem, "segmentation fault". I've raise another issue "Problem about the FWVideoRender". If you have spare time, will you please have a look at it? I'm really confused about this problem... Thanks~~
Wu Yu

[sdhays]

Some questions:
Have you tried this on many different AVIs made from different sources? Do those AVIs work flawlessly in simple_player? What compiler are you using? Are you dynamically or statically linking? Is there any way for you to try this on a different computer?

I've tried this on both a Windows XP system compiled with both the Intel Compiler 9.0 and the Microsoft compiler 7.1.xxx as well as gcc-3.4.5 on Red Hat 9. All of my builds use static linking, and they all work as expected. I haven't tried dynamic linking, so while it shouldn't be a problem, I just don't know.

From your description, it doesn't sound like the problem is in the renderer; it sounds like no frame is being decoded for the renderer to render and since it doesn't check for that possibility, it seg faults. That's what was happening earlier for me. So the root of the problem is probably farther up the pipeline; the AVI splitter seems most likely to me, especially since you've had some trouble with some of your AVI's with simple_player.

I've attached an AVI that works for me. It was encoded and multiplexed using a fairly recent CVS version of FFMpeg. If it still doesn't work for you, then I guess the next thing I'd want to try is building on a different computer if that's possible.

Good luck,
Scott

[walkdeath]

Hi Scott,
Thanks very much for your spending time on my problem~~I've tried your avi file, "segmentation fault" again.
I've tried 3 avi files on two computer, all "segmentation fault".
But all of them run well using simple player, but it played somewhat fast when using your avi file. I have not learnt much about "make" and "link mode", so I use the "build32.sh" and "Makefile.lnx" supplied by the ipp_sample. I think I use gcc 4.0 and "static linkage mode", but I'm not very sure~~~
Shall we have a chat on the net if your have spare time? My MSN is kilua_wuyu@hotmail.com or we can use another way.
Thank you very much~~

Wu Yu

[sdhays]

It runs faster in simple_player because it has no audio, so it plays as fast as it decodes. Specifying "-N" on the commandline should make it play at normal speed.

The default build for the IPP samples uses shared linking. Use ldd to find out if your binary has dependencies on the IPP (core) libraries if you're not sure if you're using static linking or or dynamic linking. I think I made a post mentioning some minor modifications for making the static linking work properly for simple _player using GCC earlier on this forum.

GCC 4.0 could possibly be a problem. It's not officially supported with the current version of the IPP libraries. GCC 3.4.5 is the latest supported version, I believe, so you may want to try that. Or you could download a trial version of the Intel compiler to see if that solves the problem.

Feel free to email me. I don't have a lot of time (and I'm running out of ideas for what might be wrong), but if I can help, I will.

Scott

[Vladimir_Dudnik]

Hi Scott,

no, simple_player can't play raw YUV file. It is not difficult to write such utility (does not matter with using UMC or not) or you can look for some third-party YUV viewer, I was able to find several in google

Regards,
Vladimir

[klend]

---

vdudnik wrote:

I've attached MSVC project for your convenience, please let me know if it work for you.

Note, you need to unzip umc_mpeg4_dec folder to UMCapplications folder. And you need to pre-build UMC static libraries as this project depends from prebuild libraries for similicity.

Regards,
Vladimir

---

Hi Vladimir,
Thanks for this example, together with the SetVideoParameters call from a later post I got this to work rather nicely. however, I do have a question regarding the call to mpeg4_dec.GetFrame(&in, &out):

What if I want to access the YV12 data in the 'out' buffer (instead of just rendering them with FWVideoRenderer)?
I do not seem to find a nice interface to the YUV parts except the out.m_lpDest[0] .. out.m_lpDest[2] pointers.
Do I overlook the obvious? I did look at the NullVideoRenderer and thought about changing that a bit, but before I do, I want to know if I'm not missing something, as I think I do.

Please enlighten me,
Thanks in advance,
Hendrik

Message Edited by klend on 03-14-2006 12:55 PM

[Vladimir_Dudnik]

Hi Hendrik

inthat version there is no methods to access to VideoData's internal data members, so you can add such or use direct access.

Regards,
Vladimir

[jordimitja]

Hi everybody:

I tested the wrong avi (bad_video.zip) with the new version IPP v5.1 and the results are the same. The simple_player fails and I cant reproduce this avi. However I solved the problem, its a problem in the avifile header, its wrong in the chunk "vids", I missed the fourcc handle necessary to decompress the video stream. Now it works correctly.

Thanks you very much !!!

Another question, Im using the MPEG-4 Decoder sample and I would like to use RGB24 output format but I see that the images are flipped. To fix this and obtain the images correctly I need to call to the function ippiMirror_8u_C3R after decompress the image:

IppStatus status = ippStsOk;
Ipp8u* pSrc[1];
pSrc[0] = (unsigned char*) out_.GetBufferPointer();
int srcStep[1];
srcStep[0] = pitch_;
IppiSize roiSize[1];
roiSize[0].width = image_width_;
roiSize[0].height = image_height_;

Ipp8u* pDst_mirror[1];
pDst_mirror[0] = ret.buffer.buffer();
IppiAxis flip_axis = ippAxsHorizontal;

status = ippiMirror_8u_C3R(pSrc[0], srcStep[0], pDst_mirror[0], srcStep[0],roiSize[0], (IppiAxis) flip_axis);

I would like to known if is it possible to configure any decoder option or any flags to avoid this operation? Im very worried about the performance impact because I need to decompress a lot of stream (up to 16). I need to use RGB24 because sometimes I must to realize any post-processing in decoded image, I known that the fastest decoder output format is YV12.

Thank you very much in advance !!

Jordi.

[Vladimir_Dudnik]

Hello Jordi,

how did you get flipped image? UMC decoder does not rotate video frame!

Regards,
Vladimir

[jordimitja]

Hi Vladimir,

The frames that Im decoding are generated compressing the frames with XVID codec o DivX Pro Codec from a video capture card based in chip Bt878. If I store in BMP file format or I display a decompressed image in RGB24 format captured I can see the image correctly. If I display the frames decoded using XVID decoder or DivX decoder the images are not flipped.

Any ideas ?

Thank you very much in advance !!

Regards,

Jordi.

[Vladimir_Dudnik]

Remember, that BMP support both top-left and bottom-left image orientation. You control this by changing sign for BITMAPINFOHEADER biHeight variable. Positive valuse means bottom-left orientation and so image will be flipped by Win32 API draw function. You need to set it to negative value to see image not flipped.

Note, encoders (as far as I know, any MPEG like) support only top-left image orientation.

Vladimir

[jordimitja]

Hi everybody,

I see where its the problem with the decoder. The problem its in the color conversion routines IPP from RGB to YUV. I read in a web page the next sentence:

"Color conversion from YUV 420 to RGB results in an upside-down image:
In some imaging pipelines, the interpretations of YUV images and RGB images are flipped relative to each other. The Intel IPP YUVToRGB function does not flip the image, but provides routines to perform the flip operation. One solution is to apply the function ippiMirror to flip the result before or after calling the YUVToRGB color conversion function. "

If Im using YUY2 from YV12 it works correctly and the image isnt inverted.

Regards,

Jorge.

[Vladimir_Dudnik]

Hi Jorge,

if you use Windows BITMAP structure to represent/display decoded frame you can use negative height to allow Win32 API make flip for you during rendering

Regards,
Vladimir