Deinterlacing with Encoding initialization

[Can_O_]

Hi,

I have a problem that I couldn't solve for couple of days. Unfortunately, I am using rather old version of media sdk. (R2 Linux server version, Ubuntu 12.04 on 3rd gen i7 cpu) Nonetheless, as far as I can see, the usage of deinterlacing (except inv. telecine) didn't change much from that version.

Here is my case, I have a raw YUY2 frames acquired from a frame grabber source which provides PAL 50i (720x576) frames. The interlaced fields are appended sequentially inside a frame memory as in the below picture where red is top field and blue is bottom field. This memory structure of thee acquired frame belongs to the grabber.

I couldn't come up a correct parameters/approach to enable deinterlacing for this case. I am not sure whether VPP parameters (esp. input height/width) are correct. Moreover, how should I feed this frame into the vpp->RunFrameVPPAsync(...). Should I have to separate it into two different surfaces and feed them sequential or should they be stay like this?

Well, in fact, I have tried many different cases such as seq. feed, cropped etc. etc. But none of them gave satisfying results. Either the output frames stayed like above picture or the resulting stream flickers a lot on the edges. What is the expected flow of deinterlacing? (I have read tutorials, sample codes, and forum entries but none of them explains the actual flow)

Here is my parameter initialization code:

	// ...	

	vppParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY;
	vppParams.vpp.In.FourCC = MFX_FOURCC_YUY2;
	vppParams.vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV422;
	vppParams.vpp.In.PicStruct = MFX_PICSTRUCT_FIELD_TFF;
	vppParams.vpp.In.Width = ALIGN16(720);
	vppParams.vpp.In.Height = ALIGN32(576);
	vppParams.vpp.In.FrameRateExtN = 25;
	vppParams.vpp.In.FrameRateExtD = 1;
	vppParams.vpp.In.CropX = 0;
	vppParams.vpp.In.CropY = 0;
	vppParams.vpp.In.CropW = 720;
	// should be height / 2 ???
	vppParams.vpp.In.CropH = 576;

	// copy in to out and change couple of parameters
	memcpy(&vppParams.vpp.Out, &vppParams.vpp.In, sizeof(mfxFrameInfo));
	vppParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
	vppParams.vpp.Out.Width = ALIGN16(720);
	vppParams.vpp.Out.Height = ALIGN16(576);
	vppParams.vpp.Out.FourCC = MFX_FOURCC_NV12;
	vppParams.vpp.Out.ChromaFormat = MFX_CHROMAFORMAT_YUV420;

	// disable filters but enable deinterlacing
	vppDoNotUse.Header.BufferId = MFX_EXTBUFF_VPP_DONOTUSE;
	vppDoNotUse.Header.BufferSz = sizeof(mfxExtVPPDoNotUse);
	vppDoNotUse.NumAlg = 4;
	if(!vppDoNotUse.AlgList)
		vppDoNotUse.AlgList = new mfxU32[vppDoNotUse.NumAlg];

	// Disable default VPP features
	vppDoNotUse.AlgList[0] = MFX_EXTBUFF_VPP_DENOISE;
	vppDoNotUse.AlgList[1] = MFX_EXTBUFF_VPP_SCENE_ANALYSIS;
	vppDoNotUse.AlgList[2] = MFX_EXTBUFF_VPP_DETAIL;
	vppDoNotUse.AlgList[3] = MFX_EXTBUFF_VPP_PROCAMP;

	// this is a std::vector
	vppExtParams.clear();
	vppExtParams.push_back(reinterpret_cast<mfxExtBuffer*>(&vppDoNotUse));
	if(vppParams.vpp.In.PicStruct != MFX_PICSTRUCT_PROGRESSIVE)
	{
		vppDeinterlacing.Header.BufferId = MFX_EXTBUFF_VPP_DEINTERLACING;
		vppDeinterlacing.Header.BufferSz = sizeof(mfxExtVPPDeinterlacing);
		vppDeinterlacing.Mode = MFX_DEINTERLACING_ADVANCED;
		vppExtParams.push_back(reinterpret_cast<mfxExtBuffer*>(&vppDeinterlacing));
	}

	vppParams.ExtParam = &vppExtParams[0];
	vppParams.NumExtParam = vppExtParams.size();

// setup encoding parameters
	encodeParams.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY;
	encodeParams.mfx.CodecId = MFX_CODEC_AVC;
	encodeParams.mfx.TargetUsage = MFX_TARGETUSAGE_BEST_SPEED;
	encodeParams.mfx.EncodedOrder = 0;
	encodeParams.mfx.RateControlMethod = MFX_RATECONTROL_CBR;	
	encodeParams.mfx.FrameInfo.FourCC = MFX_FOURCC_NV12;
	encodeParams.mfx.FrameInfo.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
	encodeParams.mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
	encodeParams.mfx.TargetKbps = getBitRate(720, 576, 25, MFX_TARGETUSAGE_BEST_SPEED);
	encodeParams.mfx.FrameInfo.Width = ALIGN16(720);
	encodeParams.mfx.FrameInfo.Height = ALIGN16(576);
	encodeParams.mfx.FrameInfo.FrameRateExtN = 25;
	encodeParams.mfx.FrameInfo.FrameRateExtD = 1;
	encodeParams.mfx.FrameInfo.CropX = 0;
	encodeParams.mfx.FrameInfo.CropY = 0;
	encodeParams.mfx.FrameInfo.CropW = 720;
	encodeParams.mfx.FrameInfo.CropH = 576;

	//...

Thanks in advance :)

[Bjoern_B_Intel]

Hi Can,

The version you are using is pretty old and we discontinued with Ubuntu support, while we moved to CentOS 7.1 as a gold OS only.

While there are a lot of changes, bug fixes, and improvements since R2 (especially on VPP), I recommend to update to latest Intel® Media Server Studio version and to reproduce the issue you are facing on CentOS.

Please let me know how that worked out to be able to further assist you.

Best,

Bjoern

[Bjoern_B_Intel · Accepted solution]

Hi Can,

What I overlooked is that you are trying to apply VPP deinterlacing filter to a video in YUY2 format (YUV422) according to your shared code. But our VPP-deinterlacer is only working on NV12 input format as documented the Intel® Media Server Studio Manual in table 4, page 8.

Starting with latest Intel® Media Server Studio release, I would do the following:

1. Convert the YUY2 video into NV12 format

2. Deinterlace NV12 video

3. AVC encode the deinterlaced video

Supported color conversions are listed in Intel® Media Server Studio Manual in table 2, page 7 and is just another filter to add.

Let me know if this helps.

Best,

Bjoern

[Can_O_]

Hi Bjoern,

Thanks for the reply and sorry for my late response.

SDK documention of R2 version doesn't have a table like Table 4 and doesn't mention about this restriction about supported formats for filters. I believe my problem is this. I couldn't try it quickly and tell you the results but I will write the results definitely.

However, Table 4 (and the solution you explained) is also aroused another question for me. Is it possible to do color conversion and deinterlacing in one chain or should I have to use two (pass) VPPs. Because initially, I though that one VPP will do the color conversion automatically and feed it to deinterlacing afterwards as a chain of operations.

As for the version comment in your previous post, is it possible to use SDK larger than R2 (maybe R3) on 3rd gen CPU (besides Ubuntu stuff)? Because our platform is an embedded module (rugged COM-Express module) and upgrade/modification is not an option. (at least for the moment)

Regards,

Can

[Bjoern_B_Intel]

Hi Can,

Yes, latest Intel® Media Server Studio version will also support older HW as well as Ubuntu. For installation, look at the Intel® Media Server Studio 2015 - Getting Started Guide. The “Generic Steps” section explains how to install on none CentOS OSes. We call it 2^nd OS support which will continue moving forward. All newer MSDKs will be backward compatible and support older CPUs while we Keep adding newer CPUs.

As deinterlacing is implemented using NV12 color format internally, you need to convert from YUY2 video to NV12 first. So it is a two pass chain as you said.

Best,

Bjoern

[Can_O_]

Hi Bjoern,

I should have missed the backward compatibility remarks (seeing supports 4th, 5th remarks thought that it does not support 3rd gen) so I should be able to use R6 as well. This is really really good news for me.

I have seen the newer installation guides. I believe that I could install on Ubuntu (fingers crossed :))

Thanks for your replies. You saved me a lot.

Regards,

[Bjoern_B_Intel]

Can,

Let me point you also to an article one of my colleagues wrote on 2nd OS including Ubuntu. Just in case: https://software.intel.com/en-us/articles/how-to-setup-media-server-studio-on-secondary-os-of-linux

Best,

Bjoern

[Can_O_]

Thanks for the cool article

[Bjoern_B_Intel]

Hi Can,

One last thing: A friendly colleague (Surbhi) pointed me to the MSDK release notes, saying:

"Intel platforms are not supported: Intel Core processors earlier than 4th Generation, Intel Celeron®, Intel Pentium® and Intel Atom™"

So you might want to consider when upgrading to latest Intel® Media Server Studio as 3^rd generation Intel processor is not supported.

Best,

Bjoern

[Can_O_]

Hi Bjoern,

So my initial decision of using old release is correct. After your post, I've thought that there is a missing information in the documentation. (or some insider information about backward compatibility :) )

This is so unfortunate. By the way, I still didn't have time to test the color conversion configuration. (because of other software problems needs to be solved)

Thanks, 

Can

[Bjoern_B_Intel]

Hi Can,

This is OK to me. Let me close this thread here.

Best,

Bjoern