VPP using way too much memory

[Ike_S_]

I am trying to encode 4K video using the H265/HEVC encoder plugin. I discovered the VPP process uses a large amount of memory.

The input video is 3840 x 2160 YV12. I discovered that when I use VPP to convert 3840 x 2160 YV12 -> 3840 => 2160 NV12, MFXVideoVPP_Init allocates 250MB of memory!

I set AsyncDepth to 2 to reduce the number of necessary memory. In certain situations, if the process allocated 250MB prior to calling MFXVideoVPP_Init, MFXVideoVPP_Init fails with MFX_ERR_MEMORY_ALLOC

I am asking the engine to convert up to 2 buffers of 12MB each. YV12->NV12 is also a simple conversion. So 250MB of memory for this conversion seems too much.

I have another thread in this forum in for a different kind of error occuring during encoding. I can sort of work around those problems setting AsyncDepth to 2 and using fewer frames. But in some situations, MFXVideoVPP_Init fails and conversion doesn't start at all.

[Surbhi_M_Intel]

Hi,

Thank you for reporting this. Allocating 250MB of memory seems too high for initializing VPP. I wasn't able to replicate this behavior. Can you please give us a reproducer code and point how you are checking the memory it is allocating ?
"In certain situations, if the process allocated 250MB prior to calling MFXVideoVPP_Init, MFXVideoVPP_Init fails with MFX_ERR_MEMORY_ALLOC"
Can you please explain these situations ? Is it seen with specific input file?
 

Thanks,
-Surbhi

[Ike_S_]

I created a DirectShow filter based on the DirectShow filter example in a prior version of the Intel Media SDK. The out of memory occurrs on a line like this:
        sts = m_pmfxVPP->Init(&m_mfxVppParams);

You can reproduce the 250MB allocation on the VPP init occurrence with your sample_encode. Do the following:

1) build sample_encode from the source in a debug configuration
2) correct the sample in <instdir>\samples\EncodeSample\sample_encode\src\pipeline_encode.cpp function CEncodingPipeline::InitMfxVppParams as follows:
   a) replace this line
        m_mfxVppParams.vpp.In.FourCC    = MFX_FOURCC_NV12;
       with
        m_mfxVppParams.vpp.In.FourCC    = pInParams->ColorFormat;
   b) and add this line after MSDK_MEMCPY_VAR(m_mfxVppParams.vpp.Out,&m_mfxVppParams.vpp.In, sizeof(mfxFrameInfo)):
        m_mfxVppParams.vpp.Out.FourCC    = MFX_FOURCC_NV12;
3) run the sample with these parameters: h265 -hevc_encoder_plugin 0 -i d:\i\h265\3840x2160.yv12 -o d:\i\h265\3840x2160.265 -w 3840 -h 2160
4) Set a breakpoint in <INSTDIR>\samples\EncodeSample\sample_encode\src\pipeline_encode.cpp on this line
        sts = m_pmfxVPP->Init(&m_mfxVppParams);
5) run the sample and check the memory allocated for your task before this line and after.

In my case, the memory allocated for sample_encode.exe was 241MB prior to calling m_pmfxVPP->Init and 489MB after m_pmfxVPP->Init returns. So m_pmfxVPP->Init allocated 248MB of memory. In sample_encode, the call succeeds. In my filter, sometimes it succeeds and sometimes it fails every time.

The changes in the step #2 is to make sure you use a VPP for YV12 -> NV12 colorspace conversion. Frankly, I don't know how this sample would work if you gave it YV12 input, because the sample hardcodes NV12 as the input even when the input data is YV12. (But that's a separate issue).

My question is this: can you reproduce this 250MB memory jump in VPP Init? Does that seem necessary to you? YV12->NV12 is a simple color space conversion and should not require that much memory.

[Ike_S_]

I should mention that in step #4 above, you should look in the function CEncodingPipeline::ResetMFXComponents(sInputParams* pParams) from pipeline_encode.cpp. That function contains the call to m_pmfxVPP->Init

[Surbhi_M_Intel]

Hi,

Thank you for the clear description. Can you please send me the input as well? If you are using any commonly used input which is available on web, let me know from where i can download. Till now i am not able to reproduce the issue, so i was wondering may be with the same input i will be able to.

-Surbhi