VPP uses too much memory

mskmskmsk · ‎12-21-2017

I am trying to convert RGB4 to NV12 by using VPP before encoding to h264. Input source is VGA(640x480) and initializing VPP ends up with allocating about 50MB memory. I think this is too much and would like to know if there are any ways to save memory.

Thanks

[notes]
my application is:
* 32bit
* uses only system memory.

[vpp parameter]
    // Disable default VPP operations
    mfxExtVPPDoNotUse extDoNotUse;
    memset(&extDoNotUse, 0, sizeof(mfxExtVPPDoNotUse));
    extDoNotUse.Header.BufferId = MFX_EXTBUFF_VPP_DONOTUSE;
    extDoNotUse.Header.BufferSz = sizeof(mfxExtVPPDoNotUse);
    extDoNotUse.NumAlg = 4;
    extDoNotUse.AlgList = new mfxU32[extDoNotUse.NumAlg];
    // turn off denoising (on by default)
    extDoNotUse.AlgList[0] = MFX_EXTBUFF_VPP_DENOISE;
    // turn off scene analysis (on by default)
    extDoNotUse.AlgList[1] = MFX_EXTBUFF_VPP_SCENE_ANALYSIS;
    // turn off detail enhancement (on by default)
    extDoNotUse.AlgList[2] = MFX_EXTBUFF_VPP_DETAIL;
    // turn off processing amplified (on by default)
    extDoNotUse.AlgList[3] = MFX_EXTBUFF_VPP_PROCAMP;

    // Add extended VPP buffers
    mfxExtBuffer* extBuffers[1];
    extBuffers[0] = (mfxExtBuffer*) & extDoNotUse;

    // Initialize VPP parameters
    mfxVideoParam VPPParams;
    memset(&VPPParams, 0, sizeof(VPPParams));
    // Input data
    VPPParams.vpp.In.FourCC = MFX_FOURCC_RGB4;
    VPPParams.vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    VPPParams.vpp.In.CropX = 0;
    VPPParams.vpp.In.CropY = 0;
    VPPParams.vpp.In.CropW = 640;
    VPPParams.vpp.In.CropH = 480;
    VPPParams.vpp.In.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
    VPPParams.vpp.In.FrameRateExtN = 5;
    VPPParams.vpp.In.FrameRateExtD = 1;
    VPPParams.vpp.In.Width = 640;
    VPPParams.vpp.In.Height = 480t;
    // Output data
    VPPParams.vpp.Out.FourCC = MFX_FOURCC_NV12;
    VPPParams.vpp.Out.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    VPPParams.vpp.Out.CropX = 0;
    VPPParams.vpp.Out.CropY = 0;
    VPPParams.vpp.Out.CropW = 640;
    VPPParams.vpp.Out.CropH = 480;
    VPPParams.vpp.Out.PicStruct = MFX_PICSTRUCT_PROGRESSIVE;
    VPPParams.vpp.Out.FrameRateExtN = 5;
    VPPParams.vpp.Out.FrameRateExtD = 1;
    VPPParams.vpp.Out.Width = 640
    VPPParams.vpp.Out.Height = 480;
    VPPParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY | MFX_IOPATTERN_OUT_SYSTEM_MEMORY;
    VPPParams.ExtParam = extBuffers;
    VPPParams.NumExtParam = 1;
    VPPParams.AsyncDepth = 1;

[mediasdk_sys_analyser result]
Intel(R) Media Server Studio 2017 - System Analyzer (64-bit)

The following versions of Media SDK API are supported by platform/driver
[opportunistic detection of MSDK API > 1.23]:

        Version Target Supported       Dec     Enc
        1.0     HW      Yes             X       X
        1.0     SW      Yes             X       X
        1.1     HW      Yes             X       X
        1.1     SW      Yes             X       X
        1.2     HW      Yes             X       X
        1.2     SW      Yes             X       X
        1.3     HW      Yes             X       X
        1.3     SW      Yes             X       X
        1.4     HW      Yes             X       X
        1.4     SW      Yes             X       X
        1.5     HW      Yes             X       X
        1.5     SW      Yes             X       X
        1.6     HW      Yes             X       X
        1.6     SW      Yes             X       X
        1.7     HW      Yes             X       X
        1.7     SW      Yes             X       X
        1.8     HW      No
        1.8     SW      Yes             X       X
        1.9     HW      No
        1.9     SW      Yes             X       X
        1.10    HW      No
        1.10    SW      Yes             X       X
        1.11    HW      No
        1.11    SW      Yes             X       X
        1.12    HW      No
        1.12    SW      Yes             X       X
        1.13    HW      No
        1.13    SW      Yes             X       X
        1.14    HW      No
        1.14    SW      Yes             X       X
        1.15    HW      No
        1.15    SW      Yes             X       X
        1.16    HW      No
        1.16    SW      Yes             X       X
        1.17    HW      No
        1.17    SW      Yes             X       X
        1.18    HW      No
        1.18    SW      Yes             X       X
        1.19    HW      No
        1.19    SW      Yes             X       X
        1.20    HW      No
        1.20    SW      Yes             X       X
        1.21    HW      No
        1.21    SW      Yes             X       X
        1.22    HW      No
        1.22    SW      Yes             X       X
        1.23    HW      No
        1.23    SW      Yes             X       X

Graphics Devices:
        Name                                         Version             State
        Intel(R) HD Graphics 4600                    10.18.10.3412       Active

System info:
        CPU:    Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
        OS:     Microsoft Windows 7 Professional
        Arch:   64 ???

Installed Media SDK packages (be patient...processing takes some time):
        Intel(R) Media Server Studio 2017 R3 ? Software Development Kit

Installed Media SDK DirectShow filters:

Installed Intel Media Foundation Transforms:
        Intel(R) Hardware M-JPEG Decoder MFT : {00C69F81-0524-48C0-A353-4DD9D54F9A6E}
        Intel(R) Hardware VC-1 Decoder MFT : {059A5BAE-5D7A-4C5E-8F7A-BFD57D1D6AAA}
        Intel(R) Hardware H.264 Decoder MFT : {45E5CE07-5AC7-4509-94E9-62DB27CF8F96}
        Intel(R) Hardware MPEG-2 Decoder MFT : {CD5BA7FF-9071-40E9-A462-8DC5152B1776}
        Intel(R) Quick Sync Video H.264 Encoder MFT : {4BE8D3C0-0515-4A37-AD55-E4BAE19AF471}
        Intel(R) Hardware Preprocessing MFT : {EE69B504-1CBF-4EA6-8137-BB10F806B014}

Jiandong_Z_Intel · ‎12-27-2017

Hi There,

VPP will store surface in the memory, video memory also be taken from system memory, and usually the raw data RGB/NV12 is big.

Thanks,

Zachary

Seunghyuk_P_Intel · ‎01-08-2018

Hi mskmskmsk,

Could you post code snippet to take a look at, especially from QueryIOSurf to init, memory allocation part?

Regards,

Peter.

mskmskmsk · ‎01-09-2018

Jiandong Z. (Intel) wrote:

Hi There,

VPP will store surface in the memory, video memory also be taken from system memory, and usually the raw data RGB/NV12 is big.

Thanks,

Zachary

Thank you for your reply. I am just wondering the size of rgba32(640x480) is only 1.17MByte and how many surfaces are needed just to convert colors...

mskmskmsk · ‎01-09-2018

SEUNGHYUK P. (Intel) wrote:

Hi mskmskmsk,

Could you post code snippet to take a look at, especially from QueryIOSurf to init, memory allocation part?

Regards,

Peter.

Thank you for your reply. and here is my code snippet

    vpp = new MFXVideoVPP(session);
    mfxFrameAllocRequest VPPRequest[2];
    memset(&VPPRequest, 0, sizeof(mfxFrameAllocRequest) * 2);
    vpp->QueryIOSurf(&VPPParams, VPPRequest);

    // Determine the required number of surfaces for VPP input and for VPP output (encoder input)
    int surfaceNumber1 = VPPRequest[0].NumFrameSuggested;
    int surfaceNumber2 = EncRequest.NumFrameSuggested + VPPRequest[1].NumFrameSuggested;

    // Allocate surfaces for encoder
    // - Width and height of buffer must be aligned, a multiple of 32
    // - Frame surface array keeps pointers all surface planes and general frame info
    mfxU16 width = (mfxU16) MSDK_ALIGN32(VPPRequest[0].Info.Width);
    mfxU16 height = (mfxU16) MSDK_ALIGN32(VPPRequest[0].Info.Height);
    mfxU8 bitsPerPixel = 32;
    mfxU32 surfaceSize = width * height * bitsPerPixel / 8;
    unsigned char* surfaceBuffer1 = (unsigned char*) new mfxU8[surfaceSize * surfaceNumber1];
    memset(surfaceBuffer1, 0, surfaceSize * surfaceNumber1);

     // Allocate surface headers (mfxFrameSurface1) for encoder
    mfxFrameSurface1** ppSurfaces1 = new mfxFrameSurface1 *[surfaceNumber1];

    for (int i = 0; i < surfaceNumber1; i++)
    {
            ppSurfaces1 = new mfxFrameSurface1;
            memset(ppSurfaces1, 0, sizeof(mfxFrameSurface1));
            memcpy(&(ppSurfaces1->Info), &(VPPParams.vpp.In), sizeof(mfxFrameInfo));
            ppSurfaces1->Data.B = &surfaceBuffer1[surfaceSize * i];
            ppSurfaces1->Data.G = &surfaceBuffer1[surfaceSize * i + 1];
            ppSurfaces1->Data.R = &surfaceBuffer1[surfaceSize * i + 2];
            ppSurfaces1->Data.A = &surfaceBuffer1[surfaceSize * i + 3];
            ppSurfaces1->Data.Pitch = width * 4;
    }

    // Initialize the Media SDK VPP
    vpp->Init(&VPPParams);

Seunghyuk_P_Intel · ‎02-01-2018

Hi mskmskmks,

First of all, we need to check surfaceNumber1, 2 and see how many surfaces are to be allocated for this process.

Actually, one 640*480*4 memory size is about 1MB, so, maybe there's another area which is consuming memory also, not MediaSDK.

Second thing,

Why don't you use GPU memory?

You are using system memory (CPU).

If you use video memory(GPU), you will get better performance and will use less memory in CPU.

There's many sample code for GPU usage. Please check this site.

https://software.intel.com/en-us/intel-media-server-studio-support/code-samples

Regards,

Peter.