Copying Externally Decoded Frame to MSDK Surface

[babgvant]

I'm having trouble copying an externally decoded image (ffmpeg) to a surface for further VPP & encode processing. The relevant code bits are below. Any assistance would be greatly appreciated.

This code block if from DecodeOneFrame (in modified version of sample_multi_transcode app), changed to make it possible to use an 3rd party video decoder.

// get YUV pointers
sts = m_pMFXAllocator->Lock(m_pMFXAllocator->pthis, pmfxSurface->Data.MemId, &(pmfxSurface->Data));
//MSDK_BREAK_ON_ERROR(sts);
sts = m_pBSProcessor->GetDecodedSurface(pmfxSurface);    // calls into the next code block    
if(sts == MFX_ERR_NONE)
    pExtSurface->pSurface = pmfxSurface;

sts = m_pMFXAllocator->Unlock(m_pMFXAllocator->pthis, pmfxSurface->Data.MemId, &(pmfxSurface->Data));

This code bit decodes the frame, converts it to NV12 targeting the surface's Y + UV pointers. The data is copied over w/o a recognizable error.

//some code to parse/decode unsupported video codecs

...

uint8_t *dst[4] = {NULL};
int dstStride[4] = {0};

//set the destination stride
dstStride[0] = dstStride[1] = pmfxSurface->Data.Pitch;
dstStride[2] = dstStride[3] = 0;

//assign surface pointers to destination buffer array
dst[0] = pmfxSurface->Data.Y;
dst[1] = pmfxSurface->Data.UV;
dst[2] = dst[3] = NULL;                

// Get a SwsContext
m_pSwsContext = sws_getCachedContext(
    m_pSwsContext,
    aFrame->width,
    aFrame->height,
    (AVPixelFormat)aFrame->format,
    aFrame->width,
    aFrame->height,
    AV_PIX_FMT_NV12,
    SWS_BILINEAR /*| SWS_PRINT_INFO*/,
    NULL, NULL, NULL);

if(m_pSwsContext) //make sure we have a SwsContext
{
    // Perform pixel format conversion
    ret = sws_scale(m_pSwsContext, aFrame->data, aFrame->linesize, 0, aFrame->height, dst, dstStride);
    if(ret < 0)
    {
        av_log(NULL, AV_LOG_ERROR, "Error converting to NV12\n");
        sts = MFX_ERR_UNDEFINED_BEHAVIOR;
        goto exit;
    }
    gotFrame = true;
}
else
{
    av_log(NULL, AV_LOG_ERROR, "Error getting SwsContext\n");
    sts = MFX_ERR_UNDEFINED_BEHAVIOR;
    goto exit;
}

This block is where the surface goes after being unlocked for VPP. The call fails returning -16 (MFX_ERR_UNDEFINED_BEHAVIOR).

// pSurfaceIn is the surface Y + UV were copied to previously
sts = m_pmfxVPP->RunFrameVPPAsync(pSurfaceIn, pmfxSurface, NULL, &pExtSurface->Syncp); 
//sts = -16

Thanks

[babgvant]

Interestingly this code actually works, just not for the OGG sample I was testing. So I'm leaning towards an initialization issue (maybe because the sample is 560x320).

[Petter_L_Intel]

Hi,

I'm not able to determine the issue using the information you provided. Likely an init or copy issue as you stated in your last post.

If you are just planning to use Media SDK VPP and Encode components you may be better off using the "sample_encode" sample as a starting point instead of "sample_multi_transcode" sample. Alternatively you can also look at the sample that does VPP+encode sample, part of the Media SDK tutorial.

Regards,
Petter 

[babgvant]

Thanks. I need the whole pipeline that sample_multi_transcode provided (and it's too late to start over again :)), just trying to add support for CODECs that the MSDK doesn't support natively to the decode side.

After some more testing I'm pretty confident that it's not related to this code path, and something in the way I'm manually setting up the mfxVideoParam when a non-MSDK decoder is used (see below) since other samples (even other OGG samples) with more normal width/height work properly.

    decParams->mfx.FrameInfo.AspectRatioH = 0;// m_pInVideoCodecContext->sample_aspect_ratio.num;
    decParams->mfx.FrameInfo.AspectRatioW = 0;// m_pInVideoCodecContext->sample_aspect_ratio.den;

    decParams->mfx.FrameInfo.Width = MSDK_ALIGN16(m_pInVideoCodecContext->coded_width);
    decParams->mfx.FrameInfo.Height = MSDK_ALIGN16(m_pInVideoCodecContext->coded_height);

    decParams->mfx.FrameInfo.CropW = m_pInVideoCodecContext->width;
    decParams->mfx.FrameInfo.CropH = m_pInVideoCodecContext->height;

    if(m_pInVideoCodecContext->field_order != AV_FIELD_UNKNOWN)
        decParams->mfx.FrameInfo.PicStruct = m_pInVideoCodecContext->field_order;
    else
        decParams->mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE; //assume the file is progressive

    decParams->vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    decParams->vpp.In.FourCC = MFX_FOURCC_NV12;

    decParams->vpp.In.AspectRatioH = 0;//m_pInVideoCodecContext->sample_aspect_ratio.num;
    decParams->vpp.In.AspectRatioW = 0;//m_pInVideoCodecContext->sample_aspect_ratio.den;

    decParams->vpp.In.Width = decParams->mfx.FrameInfo.Width;
    decParams->vpp.In.Height = decParams->mfx.FrameInfo.Height;

    decParams->vpp.In.CropW = decParams->mfx.FrameInfo.CropW;
    decParams->vpp.In.CropH = decParams->mfx.FrameInfo.CropH;

[Petter_L_Intel]

Hi,

Code looks ok. But one thing to explore may be the Height alignment. If you look at the Media SDK sample code you'll see the requirement for 32 byte alignment for interlaced content and 16 byte for progressive.

Regards,
Petter 

[babgvant]

Thanks. I'll check that out.