using VPP for NV12 to RGB4 conversion

[richard_s_2]

I started with SDK tutorial sample simple_6_decode - vpp_postproc and tried to modify it to perform a pixel format conversion to RGB4 (keeping all of its other functionality the same).  While it would seem that only a few changes are needed, my result produces a black screen. Here are the changes I made:

• Setting up VPP:

   VPPParams.vpp.Out.FourCC = MFX_FOURCC_RGB4; // formery MFX_FOURCC_NV12;

• Setting up the mfxFrameSurface data structures

    bitsPerPixel = 32;  // RGBA is 32 bits
    ...
    for (int i = 0; i < nSurfNumVPPOut; i++)
    {

        ...

        // formerly for YUV output
        //pmfxSurfaces2->Data.Y = &surfaceBuffers2[surfaceSize * i];
        //pmfxSurfaces2->Data.U = pmfxSurfaces2->Data.Y + width * height;
        //pmfxSurfaces2->Data.V = pmfxSurfaces2->Data.U + 1;

        // new
        pmfxSurfaces2->Data.R = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.R + 1;
        pmfxSurfaces2->Data.B = pmfxSurfaces2->Data.R + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.R + 3;

        pmfxSurfaces2->Data.Pitch = width;
    } 

• Replaced occurrences of WriteRawFrame() with WriteRawRGBFrame() which is exactly like LoadRawRGBFrame() except for fread is fwrite.

Am I misunderstanding how VPP's NV12 to RGB4 conversion should work?

[Petter_L_Intel]

Hi Richard,

A bit strange to hear that you are getting black frames. But I do see two things in your code that should be changed:

The order of the RGB pixels should be as it is illustrated in the Media SDK DirectX common sample code. See below.

        pmfxSurfaces2->Data.B = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.B + 1;
        pmfxSurfaces2->Data.R = pmfxSurfaces2->Data.B + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.B + 3;

and, for RGB4, width should be multiplied by 4.

        pmfxSurfaces2->Data.Pitch = width * 4;

Also, make sure your RGB4 file write routine works as expected.

Regards,
Petter 

[richard_s_2]

Thanks.  The width adjustment fixed things.

[Fernando_B_]

Hi,

I'm trying to do the same... convert a NV12 to RGB basically I have performed the same modifications as you mention, but the RGB image I get is black.

did you performed more modifications to write the RGB into a file?

This is my configuration...

Input VPP

 VPPParams.vpp.In.FourCC = MFX_FOURCC_NV12;
 VPPParams.vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
 VPPParams.vpp.In.CropX = 0;
 VPPParams.vpp.In.CropY = 0;
 VPPParams.vpp.In.CropW = _mfxDecParams.mfx.FrameInfo.CropW;
 VPPParams.vpp.In.CropH = _mfxDecParams.mfx.FrameInfo.CropH;
 VPPParams.vpp.In.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
 VPPParams.vpp.In.FrameRateExtN = 30;
 VPPParams.vpp.In.FrameRateExtD = 1;

 VPPParams.vpp.In.Width = MSDK_ALIGN16(VPPParams.vpp.In.CropW);
			 VPPParams.vpp.In.Height =
			        (MFX_PICSTRUCT_PROGRESSIVE == VPPParams.vpp.In.PicStruct) ?
			        MSDK_ALIGN16(VPPParams.vpp.In.CropH) :
			        MSDK_ALIGN32(VPPParams.vpp.In.CropH);

Output VPP

VPPParams.vpp.Out.FourCC = MFX_FOURCC_RGB4;//MFX_FOURCC_NV12;
			    VPPParams.vpp.Out.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
			    VPPParams.vpp.Out.CropX = 0;
			    VPPParams.vpp.Out.CropY = 0;
			    VPPParams.vpp.Out.CropW = VPPParams.vpp.In.CropW / 2;   // Resize to half size resolution
			    VPPParams.vpp.Out.CropH = VPPParams.vpp.In.CropH / 2;
			    VPPParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
			    VPPParams.vpp.Out.FrameRateExtN = 30;
			    VPPParams.vpp.Out.FrameRateExtD = 1;
			
			    VPPParams.vpp.Out.Width = MSDK_ALIGN16(VPPParams.vpp.Out.CropW);
			    VPPParams.vpp.Out.Height =
			        (MFX_PICSTRUCT_PROGRESSIVE == VPPParams.vpp.Out.PicStruct) ?
			        MSDK_ALIGN16(VPPParams.vpp.Out.CropH) :
			        MSDK_ALIGN32(VPPParams.vpp.Out.CropH);

VPP params for surface creation

width = (mfxU16) MSDK_ALIGN32(VPPRequest[1].Info.Width);
			height = (mfxU16) MSDK_ALIGN32(VPPRequest[1].Info.Height);
bitsPerPixel = 32;
surfaceSize = width * height * bitsPerPixel / 8;
surfaceBuffers2 = (mfxU8*) new mfxU8[surfaceSize * nSurfNumVPPOut];

To write the RGB file I use this...

for (mfxU16 i = 0; i < h; i++) {
 nBytesRead = fwrite(pSurface->Data.B + i * pSurface->Data.Pitch, 1, w * 4, fSource);
		 if ((size_t)(w * 4) != nBytesRead)
		    return MFX_ERR_MORE_DATA;
		}

something missing... I get black image... 

Thanks!!

[Surbhi_M_Intel]

Hi Fernando, 

I am able to successfully decode & do color conversion with changes mentioned by you & the one discussed above that. 
Few things to check - 
I hope you are  made the changes for VPP out surfaces for RGB(mentioned above in Petter's reply)in addition to your changes - 

       pmfxSurfaces2->Data.B = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.B + 1;
        pmfxSurfaces2->Data.R = pmfxSurfaces2->Data.B + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.B + 3;
        pmfxSurfaces2->Data.Pitch = width * 4;

Another thing is to change the function writeRGBframe() in all the stages. If you still see an issue please send us your application and input to check. 

Thanks,
Surbhi