Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3056 Discussions

Using Intel H264 Encoder MFT with SourceReader/SinkWriter

sjbaran
Novice
1,500 Views
related to: http://software.intel.com/en-us/forums/showthread.php?t=80307&o=a&s=lr

not finding a color converter to convert from YUY2 (or some other format output by the camera)to NV12
I used Intel VPP instead to do the conversion. I grabbed the IMFSample from the IMFSoureReader, got the buffer for the video frame and used VPP to convert it to NV12.

That all went smoothly and I can even save these NV12 frames to a file and play back using FFMpeg. When I try to wrap these frames into a IMFSample and pass to an encoder MFT (either Intel's or Microsoft's) they return an errorsuch asERROR_INVALID_PARAMETER.

What I'm wondering is how to properlyget the videoattributesfrom the VPP color conversionto a MFT.

Below are some pieces of code:

//-----------------------------
// Setup Sink Writer
//------------------------------

hr = m_pReader->GetCurrentMediaType((DWORD)MF_SOURCE_READER_FIRST_VIDEO_STREAM, &pType);

hr = pType->SetGUID(MF_MT_SUBTYPE,MFVideoFormat_NV12);

hr=pType->GetGUID(MF_MT_SUBTYPE,&guidSubtype);
hr=MFGetAttributeSize(pType,MF_MT_FRAME_SIZE, &vidWidth, &vidHeight);
hr=MFGetStrideForBitmapInfoHeader(guidSubtype.Data1,vidWidth,&lStride);

hr=pType->SetUINT32(MF_MT_DEFAULT_STRIDE,UINT32(lStride));

vidSampleSize=(UINT32)(vidHeight*lStride+(vidHeight>>1)*lStride);
hr=pType->SetUINT32(MF_MT_SAMPLE_SIZE,vidSampleSize);

m_pWriter->AddStream(pType2,0);
//pType2 same as pType but with MF_MT_SUBTYPE= MFVideoFormat_H264 and a bitrate set

m_pWriter->SetInputMediaType(0, pType, NULL);

//-----------------------------
// VPP Color Conversion
//------------------------------

// Video Enhancement Algorithms
pColorConvertParams->denoiseParam.factor = 1;
pColorConvertParams->denoiseParam.mode=VPP_FILTER_DISABLED;
pColorConvertParams->detailParam.factor=1;
pColorConvertParams->detailParam.mode=VPP_FILTER_DISABLED;
pColorConvertParams->procampParam.brightness=0.0;
pColorConvertParams->procampParam.contrast=1.0;
pColorConvertParams->procampParam.saturation=1.0;
pColorConvertParams->procampParam.hue=0.0;
pColorConvertParams->procampParam.mode=VPP_FILTER_DISABLED;
pColorConvertParams->vaParam.mode=VPP_FILTER_DISABLED;
pColorConvertParams->impLib = MFX_IMPL_HARDWARE;
pColorConvertParams->bd3dAlloc = true;

//input frame

pColorConvertParams->frameInfo[VPP_IN].nWidth=vidWidth;
pColorConvertParams->frameInfo[VPP_IN].nHeight=vidHeight;
pColorConvertParams->frameInfo[VPP_IN].CropX=0;
pColorConvertParams->frameInfo[VPP_IN].CropY=0;
pColorConvertParams->frameInfo[VPP_IN].CropW=vidWidth;
pColorConvertParams->frameInfo[VPP_IN].CropH=vidHeight;
pColorConvertParams->frameInfo[VPP_IN].FourCC = MFX_FOURCC_YUY2;
pColorConvertParams->frameInfo[VPP_IN].PicStruct=MFX_PICSTRUCT_PROGRESSIVE;
pColorConvertParams->frameInfo[VPP_IN].dFrameRate=30.0; //same as source

//output frame

pColorConvertParams->frameInfo[VPP_OUT].nWidth=vidWidth;
pColorConvertParams->frameInfo[VPP_OUT].nHeight=vidHeight;
pColorConvertParams->frameInfo[VPP_OUT].CropX=0;
pColorConvertParams->frameInfo[VPP_OUT].CropY=0;
pColorConvertParams->frameInfo[VPP_OUT].CropW=vidWidth;
pColorConvertParams->frameInfo[VPP_OUT].CropH=vidHeight;
pColorConvertParams->frameInfo[VPP_OUT].FourCC = MFX_FOURCC_NV12;
pColorConvertParams->frameInfo[VPP_OUT].PicStruct=MFX_PICSTRUCT_PROGRESSIVE;
pColorConvertParams->frameInfo[VPP_OUT].dFrameRate=30.0;
pColorConvertParams->requestedFramesCount=1;

//...

sts = FrameProcessor.pmfxVPP->RunFrameVPPAsync( pSurf[VPP_IN], pSurf[VPP_OUT], (VPP_FILTER_DISABLED != Params.vaParam.mode)? &extVPPAuxData : NULL, &syncPoint );

//...

//------------------------------------------------------------------
// Create a new IMFMediaSample 'pConvertedSample'
// and copy attributes from 'pSample' returned by the Source Reader
//-------------------------------------------------------------------

hr = pSample->GetUINT32(MFSampleExtension_BottomFieldFirst,&unBottomFieldFirst);
hr = pSample->GetUINT32(MFSampleExtension_CleanPoint,&unCleanPoint);
hr = pSample->GetUINT32(MFSampleExtension_Interlaced,&unInterlaced);
hr = pSample->GetUINT32(MFSampleExtension_Discontinuity,&unDiscontinuity);
hr = pSample->GetSampleTime(&hnsSampleTime);
hr = pSample->GetSampleDuration(&hnsSampleDuration);

hr = pConvertedSample->SetUINT32(MFSampleExtension_BottomFieldFirst, unBottomFieldFirst);
hr = pConvertedSample->SetUINT32(MFSampleExtension_CleanPoint, unCleanPoint);
hr = pConvertedSample->SetUINT32(MFSampleExtension_Interlaced, unInterlaced);
hr = pConvertedSample->SetUINT32(MFSampleExtension_Discontinuity, unDiscontinuity);
hr = pConvertedSample->SetSampleTime(hnsSampleTime);
hr = pConvertedSample->SetSampleDuration(hnsSampleDuration);



any help would be appreciated
thanks

0 Kudos
3 Replies
Nikolay_A_Intel
Employee
1,500 Views
Could you please be more specific, what function returns error?
0 Kudos
sjbaran
Novice
1,500 Views
The Media Foundation Tranforms return errors as seen by MFTrace. The errors indicate that there is something wrong with the IMFSample I'm passing into the transform.

I get errors with the following functions:

IMFSinkWriter::WriteSample(0,my_pSample); //if I'm using a IMFSinkWriter to setup the encoder

or
IMFTransform::ProcessInput(0,my_pSample,0);
IMFTransform::ProcessOutput(0,1,&OutputSamples,0); //if I'm trying to use the encoder directly

Note: I allocate a buffer for the output in this case. SetInputType, SetOutPutType, and ProcessInput all pass - I get the failure at ProcessOutput.

my_pSample is the sample I get from IMFSourceReader (in the YUY2 format)and then convert to NV12 using VPP functions (I called it pConvertedSample above).

errors:
CMFTransformDetours::ProcessOutput @0068FB54 failed hr=0x0x80070057 ERROR_INVALID_PARAMETER
CMFSinkWriterDetours::WriteSample @00447920 failed hr=0x80070057 ERROR_INVALID_PARAMETER

I really have two questions:
1) how do I properly set the IMFMediaType when setting up the MFT(Encoder) and properly setup the IMFSample when passing to the MFT (I gave the code for what I'm doing, but I must be missing some values)

2) how do I match up the video frame properties from the VPPparams/functionsto the attributes ofIMFMediaType and IMFSample (there are some values that I don't know, like say pixel aspect ratio, that I'm just copying from theIMFMediaType from the input device using IMFSourceReader::GetCurrentMediaType)
0 Kudos
sjbaran
Novice
1,500 Views
Looks like the answer to my second question is in the mfxVideoParam structure. I'm not setting all the values in the structure so I suppose the VPP is making some assumptions, but I can fix that.mfxFrameInfo, for example, has the aspect ratio, and so if I need it I can get it.
0 Kudos
Reply