IOPattern changed by MFXVideoVPP_Query

[Robby_S]

Greetings,

I added a VPP to resize the video frames, after the decoder and before the encoder. The pipeline worked well if I set the IOPattern to SYSTEM_MEMORY and VIDEO_MEMORY. However, when I set the IOPattern to OPAQUE_MEMORY, the call to MFXVideoVPP_QueryIOSurf() always returned error -15 (MFX_ERR_INVALID_VIDEO_PARAM).

So I tried to call MFXVideoVPP_Query() before MFXVideoVPP_QueryIOSurf() so as to verify the params, and realized that params had been changed by MFXVideoVPP_Query(). 

Below is my code snippet:

    // decide which memory to use
    switch(m_memType) {
        case SYSTEM_MEMORY:
            m_mfxVppParams.IOPattern = MFX_IOPATTERN_IN_SYSTEM_MEMORY | MFX_IOPATTERN_OUT_SYSTEM_MEMORY;
            break;
        case VIDEO_MEMORY:
            m_mfxVppParams.IOPattern = MFX_IOPATTERN_IN_VIDEO_MEMORY | MFX_IOPATTERN_OUT_VIDEO_MEMORY;
            break;
        case OPAQUE_MEMORY:
            m_mfxVppParams.IOPattern = MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY;
            break;
        default:
            msdk_printf(MSDK_STRING("  channel %u: invalid memory type !\n"), m_nChanID); 
            break;
    }
    msdk_printf(MSDK_STRING("  channel %u: vpp before query: IOPattern == 0x%0x;\n"), m_nChanID, m_mfxVppParams.IOPattern);

    sts = m_pmfxVPP->Query(&m_mfxVppParams, &m_mfxVppParams);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

    msdk_printf(MSDK_STRING("  channel %u: vpp after query: IOPattern == 0x%0x;\n"), m_nChanID, m_mfxVppParams.IOPattern);

And here is the stdout output:

  channel 0: instantiating the vpp ...
  channel 0: setting up the vpp params ...
  channel 0: vpp before query: IOPattern == 0x44;
  channel 0: vpp after query: IOPattern == 0x2;

The IOPattern change was also confirmed by the MFX tracer. Before the call to m_pmfxVPP->Query(), IOPattern in the vpp params are like this:

    in.IOPattern=UNKNOWN
    out.IOPattern=UNKNOWN

I don't understand why they were reported as "Unknown", as I had set them to be 
(MFX_IOPATTERN_IN_OPAQUE_MEMORY | MFX_IOPATTERN_OUT_OPAQUE_MEMORY).

However, after the call, they became like below, which was obviously wrong.
    in.IOPattern=MFX_IOPATTERN_IN_SYSTEM_MEMORY
    out.IOPattern=MFX_IOPATTERN_IN_SYSTEM_MEMORY

Any ideas/suggestions as to why this happened ?

Thanks,
Robby

[Robby_S]

Interesting... To ensure that opaque memory works on my platform at all, I tried one of the tutorials, simple_5_transcode_opaque, and got a segmentation fault.

Here is what the code output:

;============================================================================
 ./simple_transcode_opaque -hw -b 200 -f 30/1 ../content/test_stream.264
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Segmentation fault
;============================================================================

I am also attaching the MFX tracer's log file. Could someone take a look, and tell me what was wrong?

Thanks,

Robby

[Robby_S]

Update:

For simple_5_transcode_opaque, it looks like the segmentation fault happened near the end of the program, I added the option to generate an output file, and got this:

;============================================================================
./simple_transcode_opaque -hw -b 200 -f 30/1 ../content/test_stream.264 tmp.h264
libva info: VA-API version 0.35.0
libva info: va_getDriverName() returns 0
libva info: User requested driver 'iHD'
libva info: Trying to open /opt/intel/mediasdk/lib64/iHD_drv_video.so
libva info: Found init function __vaDriverInit_0_32
libva info: va_openDriver() returns 0
Frame number: 101
Execution time: 0.08 s (1214.58 fps)
Segmentation fault
;============================================================================

The generated output file could be played by ffplay. The new MFX tracer log file is attached.

-Robby

[Robby_S]

Judging by the 2nd MFX tracer log file I uploaded, the segmentation fault for the tutorial code apparently happened because MFXVideoDECODE_Close() and MFXVideoENCODE_Close() somehow were called twice. 

Obviously, opaque memory does work on my platform.

So now back to my original question: Why did MFXVideoVPP_Query() change my IOPattern?

-Robby

[Robby_S · Accepted solution]

It looks like a bug in MFXVideoVPP_Query (2015R6). 

In the simple_5_transcode_opaque_async_vppresize example, if I make the following change:

    // Query number of required surfaces for VPP
    mfxFrameAllocRequest VPPRequest[2];     // [0] - in, [1] - out
    memset(&VPPRequest, 0, sizeof(mfxFrameAllocRequest) * 2);
    sts = mfxVPP.Query(&VPPParams, &VPPParams);  // <---- add this line
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);
    sts = mfxVPP.QueryIOSurf(&VPPParams, VPPRequest);
    MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);

The call to QueryIOSurf() will fail with MFX_ERR_INVALID_VIDEO_PARAM, because Query() has changed the IOPattern. That can be confirmed by the MFX tracer log.

[Sravanthi_K_Intel]

Hi Robby - Thanks for the detailed analysis, I am observing similar issue when I add the Query() function. Without the Query() function, the tutorial runs fine without any issues. Are you observing correct execution without Query and invalid params with it? - I ask because I want to ensure you are not gated on this issue and can continue. Meanwhile, I will try to understand this issue and get back to you.

[Robby_S]

Hi Sravanthi - Thanks for confirming the issue. 

My code (multi-channel surveillance system) works with video memory and system memory, but not with opaque memory. I was hoping the the Query() function could help me debug. It's not helping here, but is unlikely the cause of my problem.

So yes I am sort of gated, but not by this specific issue with the Query() function.

-Robby