Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Copying Externally Decoded Frame to MSDK Surface

babgvant
Beginner
343 Views

I'm having trouble copying an externally decoded image (ffmpeg) to a surface for further VPP & encode processing. The relevant code bits are below. Any assistance would be greatly appreciated.

This code block if from DecodeOneFrame (in modified version of sample_multi_transcode app), changed to make it possible to use an 3rd party video decoder.


// get YUV pointers
sts = m_pMFXAllocator->Lock(m_pMFXAllocator->pthis, pmfxSurface->Data.MemId, &(pmfxSurface->Data));
//MSDK_BREAK_ON_ERROR(sts);
sts = m_pBSProcessor->GetDecodedSurface(pmfxSurface);    // calls into the next code block   
if(sts == MFX_ERR_NONE)
    pExtSurface->pSurface = pmfxSurface;

sts = m_pMFXAllocator->Unlock(m_pMFXAllocator->pthis, pmfxSurface->Data.MemId, &(pmfxSurface->Data));

This code bit decodes the frame, converts it to NV12 targeting the surface's Y + UV pointers. The data is copied over w/o a recognizable error.

//some code to parse/decode unsupported video codecs

...

uint8_t *dst[4] = {NULL};
int dstStride[4] = {0};

//set the destination stride
dstStride[0] = dstStride[1] = pmfxSurface->Data.Pitch;
dstStride[2] = dstStride[3] = 0;

//assign surface pointers to destination buffer array
dst[0] = pmfxSurface->Data.Y;
dst[1] = pmfxSurface->Data.UV;
dst[2] = dst[3] = NULL;                

// Get a SwsContext
m_pSwsContext = sws_getCachedContext(
    m_pSwsContext,
    aFrame->width,
    aFrame->height,
    (AVPixelFormat)aFrame->format,
    aFrame->width,
    aFrame->height,
    AV_PIX_FMT_NV12,
    SWS_BILINEAR /*| SWS_PRINT_INFO*/,
    NULL, NULL, NULL);

if(m_pSwsContext) //make sure we have a SwsContext
{
    // Perform pixel format conversion
    ret = sws_scale(m_pSwsContext, aFrame->data, aFrame->linesize, 0, aFrame->height, dst, dstStride);
    if(ret < 0)
    {
        av_log(NULL, AV_LOG_ERROR, "Error converting to NV12\n");
        sts = MFX_ERR_UNDEFINED_BEHAVIOR;
        goto exit;
    }
    gotFrame = true;
}
else
{
    av_log(NULL, AV_LOG_ERROR, "Error getting SwsContext\n");
    sts = MFX_ERR_UNDEFINED_BEHAVIOR;
    goto exit;
}

This block is where the surface goes after being unlocked for VPP. The call fails returning -16 (MFX_ERR_UNDEFINED_BEHAVIOR).


// pSurfaceIn is the surface Y + UV were copied to previously
sts = m_pmfxVPP->RunFrameVPPAsync(pSurfaceIn, pmfxSurface, NULL, &pExtSurface->Syncp);
//sts = -16

Thanks

0 Kudos
5 Replies
babgvant
Beginner
343 Views

Interestingly this code actually works, just not for the OGG sample I was testing. So I'm leaning towards an initialization issue (maybe because the sample is 560x320).

0 Kudos
Petter_L_Intel
Employee
343 Views

Hi,

I'm not able to determine the issue using the information you provided. Likely an init or copy issue as you stated in your last post.

If you are just planning to use Media SDK VPP and Encode components you may be better off using the "sample_encode" sample as a starting point instead of "sample_multi_transcode" sample. Alternatively you can also look at the sample that does VPP+encode sample, part of the Media SDK tutorial.

Regards,
Petter 

0 Kudos
babgvant
Beginner
343 Views

Thanks. I need the whole pipeline that sample_multi_transcode provided (and it's too late to start over again :)), just trying to add support for CODECs that the MSDK doesn't support natively to the decode side.

After some more testing I'm pretty confident that it's not related to this code path, and something in the way I'm manually setting up the mfxVideoParam when a non-MSDK decoder is used (see below) since other samples (even other OGG samples) with more normal width/height work properly.

    decParams->mfx.FrameInfo.AspectRatioH = 0;// m_pInVideoCodecContext->sample_aspect_ratio.num;
    decParams->mfx.FrameInfo.AspectRatioW = 0;// m_pInVideoCodecContext->sample_aspect_ratio.den;

    decParams->mfx.FrameInfo.Width = MSDK_ALIGN16(m_pInVideoCodecContext->coded_width);
    decParams->mfx.FrameInfo.Height = MSDK_ALIGN16(m_pInVideoCodecContext->coded_height);

    decParams->mfx.FrameInfo.CropW = m_pInVideoCodecContext->width;
    decParams->mfx.FrameInfo.CropH = m_pInVideoCodecContext->height;

    if(m_pInVideoCodecContext->field_order != AV_FIELD_UNKNOWN)
        decParams->mfx.FrameInfo.PicStruct = m_pInVideoCodecContext->field_order;
    else
        decParams->mfx.FrameInfo.PicStruct = MFX_PICSTRUCT_PROGRESSIVE; //assume the file is progressive

    decParams->vpp.In.ChromaFormat = MFX_CHROMAFORMAT_YUV420;
    decParams->vpp.In.FourCC = MFX_FOURCC_NV12;

    decParams->vpp.In.AspectRatioH = 0;//m_pInVideoCodecContext->sample_aspect_ratio.num;
    decParams->vpp.In.AspectRatioW = 0;//m_pInVideoCodecContext->sample_aspect_ratio.den;

    decParams->vpp.In.Width = decParams->mfx.FrameInfo.Width;
    decParams->vpp.In.Height = decParams->mfx.FrameInfo.Height;

    decParams->vpp.In.CropW = decParams->mfx.FrameInfo.CropW;
    decParams->vpp.In.CropH = decParams->mfx.FrameInfo.CropH;

0 Kudos
Petter_L_Intel
Employee
343 Views

Hi,

Code looks ok. But one thing to explore may be the Height alignment. If you look at the Media SDK sample code you'll see the requirement for 32 byte alignment for interlaced content and 16 byte for progressive.

Regards,
Petter 

0 Kudos
babgvant
Beginner
343 Views

Thanks. I'll check that out.

0 Kudos
Reply