Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

using VPP for NV12 to RGB4 conversion

richard_s_2
Beginner
1,208 Views

I started with SDK tutorial sample simple_6_decode - vpp_postproc and tried to modify it to perform a pixel format conversion to RGB4 (keeping all of its other functionality the same).  While it would seem that only a few changes are needed, my result produces a black screen. Here are the changes I made:

  • Setting up VPP:

   VPPParams.vpp.Out.FourCC = MFX_FOURCC_RGB4; // formery MFX_FOURCC_NV12;

  • Setting up the mfxFrameSurface data structures

    bitsPerPixel = 32;  // RGBA is 32 bits
    ...
    for (int i = 0; i < nSurfNumVPPOut; i++)
    {

        ...

        // formerly for YUV output
        //pmfxSurfaces2->Data.Y = &surfaceBuffers2[surfaceSize * i];
        //pmfxSurfaces2->Data.U = pmfxSurfaces2->Data.Y + width * height;
        //pmfxSurfaces2->Data.V = pmfxSurfaces2->Data.U + 1;

        // new
        pmfxSurfaces2->Data.R = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.R + 1;
        pmfxSurfaces2->Data.B = pmfxSurfaces2->Data.R + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.R + 3;


        pmfxSurfaces2->Data.Pitch = width;
    } 

  • Replaced occurrences of WriteRawFrame() with WriteRawRGBFrame() which is exactly like LoadRawRGBFrame() except for fread is fwrite.

Am I misunderstanding how VPP's NV12 to RGB4 conversion should work?

0 Kudos
4 Replies
Petter_L_Intel
Employee
1,208 Views

Hi Richard,

A bit strange to hear that you are getting black frames. But I do see two things in your code that should be changed:

The order of the RGB pixels should be as it is illustrated in the Media SDK DirectX common sample code. See below.

        pmfxSurfaces2->Data.B = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.B + 1;
        pmfxSurfaces2->Data.R = pmfxSurfaces2->Data.B + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.B + 3;

and, for RGB4, width should be multiplied by 4.

        pmfxSurfaces2->Data.Pitch = width * 4;

Also, make sure your RGB4 file write routine works as expected.

Regards,
Petter 

0 Kudos
richard_s_2
Beginner
1,208 Views

Thanks.  The width adjustment fixed things.

0 Kudos
Fernando_B_
Beginner
1,208 Views

Hi,

I'm trying to do the same... convert a NV12 to RGB basically I have performed the same modifications as you mention, but the RGB image I get is black.

did you performed more modifications to write the RGB into a file?

This is my configuration...

Input VPP

 VPPParams.vpp.In.FourCC = MFX_FOURCC_NV12;
 VPPParams.vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
 VPPParams.vpp.In.CropX = 0;
 VPPParams.vpp.In.CropY = 0;
 VPPParams.vpp.In.CropW = _mfxDecParams.mfx.FrameInfo.CropW;
 VPPParams.vpp.In.CropH = _mfxDecParams.mfx.FrameInfo.CropH;
 VPPParams.vpp.In.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
 VPPParams.vpp.In.FrameRateExtN = 30;
 VPPParams.vpp.In.FrameRateExtD = 1;
 VPPParams.vpp.In.Width = MSDK_ALIGN16(VPPParams.vpp.In.CropW);
			 VPPParams.vpp.In.Height =
			        (MFX_PICSTRUCT_PROGRESSIVE == VPPParams.vpp.In.PicStruct) ?
			        MSDK_ALIGN16(VPPParams.vpp.In.CropH) :
			        MSDK_ALIGN32(VPPParams.vpp.In.CropH);

 

Output VPP

VPPParams.vpp.Out.FourCC = MFX_FOURCC_RGB4;//MFX_FOURCC_NV12;
			    VPPParams.vpp.Out.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
			    VPPParams.vpp.Out.CropX = 0;
			    VPPParams.vpp.Out.CropY = 0;
			    VPPParams.vpp.Out.CropW = VPPParams.vpp.In.CropW / 2;   // Resize to half size resolution
			    VPPParams.vpp.Out.CropH = VPPParams.vpp.In.CropH / 2;
			    VPPParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
			    VPPParams.vpp.Out.FrameRateExtN = 30;
			    VPPParams.vpp.Out.FrameRateExtD = 1;
			
			    VPPParams.vpp.Out.Width = MSDK_ALIGN16(VPPParams.vpp.Out.CropW);
			    VPPParams.vpp.Out.Height =
			        (MFX_PICSTRUCT_PROGRESSIVE == VPPParams.vpp.Out.PicStruct) ?
			        MSDK_ALIGN16(VPPParams.vpp.Out.CropH) :
			        MSDK_ALIGN32(VPPParams.vpp.Out.CropH);

VPP params for surface creation

width = (mfxU16) MSDK_ALIGN32(VPPRequest[1].Info.Width);
			height = (mfxU16) MSDK_ALIGN32(VPPRequest[1].Info.Height);
bitsPerPixel = 32;
surfaceSize = width * height * bitsPerPixel / 8;
surfaceBuffers2 = (mfxU8*) new mfxU8[surfaceSize * nSurfNumVPPOut];

To write the RGB file I use this...

for (mfxU16 i = 0; i < h; i++) {
 nBytesRead = fwrite(pSurface->Data.B + i * pSurface->Data.Pitch, 1, w * 4, fSource);
		 if ((size_t)(w * 4) != nBytesRead)
		    return MFX_ERR_MORE_DATA;
		}

something missing... I get black image... 

Thanks!!

0 Kudos
Surbhi_M_Intel
Employee
1,208 Views

Hi Fernando, 

I am able to successfully decode & do color conversion with changes mentioned by you & the one discussed above that. 
Few things to check - 
I hope you are  made the changes for VPP out surfaces for RGB(mentioned above in Petter's reply)in addition to your changes - 

       pmfxSurfaces2->Data.B = &surfaceBuffers2[surfaceSize * i];
        pmfxSurfaces2->Data.G = pmfxSurfaces2->Data.B + 1;
        pmfxSurfaces2->Data.R = pmfxSurfaces2->Data.B + 2;
        pmfxSurfaces2->Data.A = pmfxSurfaces2->Data.B + 3;
        pmfxSurfaces2->Data.Pitch = width * 4;

Another thing is to change the function writeRGBframe() in all the stages. If you still see an issue please send us your application and input to check. 

Thanks,
Surbhi

 

0 Kudos
Reply