On-the-fly switching of VPP input crop params causes corrupted frames

[David_Pochron]

I am trying to normalize several inputs of groups of video frames into one standardized encoded output. I am trying to normalize these with the VPP, leaving the overal width and height of the input surfaces constant but changing the crop W and H values as each input's format switches.

However, after I get a free input surface to change its crop params and copy the data into it, the resulting image passed to the decoder is corrupted, usually the U and V data is scrambled and the Y channel is multiplied several times over the frame when I play back the resulting output video.

My code for the VPP phase of looks something like the following:

for (;;)
{
    if ((taskIdx = getFreeTaskIndex()) >= MFX_ERR_NONE) // get a free task slot to do this encode operation with
    {
        // Get an available input surface so we can copy our frame pixel data into it
        int surfIdxVPP = getFreeSurfaceIndex(pmfxSurfacesVPPIn, mfxResponseVPPIn.NumFrameActual);
        mfxFrameSurface1 *surfIn = pmfxSurfacesVPPIn[surfIdxVPP];
        mfxFrameInfo* pInfo = &surfIn->Info;

        // If the current input frame is not the same as the previous frame, change the VPP input surface info
        if (srcInfo->width != curSrcW  ||  srcInfo->height != curSrcH)
        {
            pInfo->CropW = curSrcW = srcInfo->width;
            pInfo->CropH = curSrcH = srcInfo->height;
        }

        // Get a pointer to the video memory to copy the source frame into the VPP
        if ((sts = DXAllocator.Lock(DXAllocator.pthis, surfIn->Data.MemId, &(surfIn->Data))) == MFX_ERR_NONE)
        {
              // Do the copy, using new/current cropW and cropH
              // .... (lots of pixel copying code goes here)

              // Unlock the surface now that we're done writing into it
              sts = DXAllocator.Unlock(DXAllocator.pthis, pmfxSurfacesVPPIn[surfIdxVPP]->Data.MemId, &(pmfxSurfacesVPPIn[surfIdxVPP]->Data));
        }

        // Start converting using the VPP; first find a place to store the output and send it to the encoder
        surfIdxEnc = getFreeSurfaceIndex(pVPPSurfacesVPPOutEnc, mfxResponseVPPOutEnc.NumFrameActual);
        for (;;)
        {
            if (MFXVideoVPP_RunFrameVPPAsync(session, surfIn, pVPPSurfacesVPPOutEnc[surfIdxEnc], NULL, &syncpVPP) == MFX_WRN_DEVICE_BUSY)
                    Sleep(10);
                else
                    break;
        }
        //MFXVideoCORE_SyncOperation(session, syncpVPP, 60000);  // debugging - sync on VPP

        // ENCODING HAPPENS HERE   
        // ...
        break;
    }
    else
    {
        // Sync on the encoder and write the bitstream if no task slots available (mostly the same as the example code)
    }

}

There is a bunch of error checking in the code as well, but I left it out here for clarity as well as the code to empty the delayed frames when it reaches the end of all the inputs.

If I uncomment the sync operation on the VPP, the code works but runs very slowly since we are waiting on the VPP every frame. If I try to run the VPP asynchronously, it causes the problem described. My guess is there is supposed to be a VPP sync in here somewhere (I am only syncing on the encoder since the encoder automatically waits for any unfinished VPP as is my understanding) but I'm not sure where to put it? I tried putting it right before I change the params on the surface but that doesn't work, and in fact most of the time the VPP sync pointer is NULL at that point.

[David_Pochron]

Never mind! I found the bug.... I just noticed I am only changing the params of ONE surface since I only check against a global "curSrcW" and "curSrcH"... The other surfaces in the task pool still had the old params which meant the pixel data would be copied incorrectly. Sorry to bother everyone, hope this post it useful to someone anyway! :-)