I've been playing with the sample_multi_transcode sample, trying to extend it so that it actually supports framerate conversion but I haven't been seeing results consistent with what I expect (frame count delivered to decoder = frames count output by encoder) when output framerate is set to 1/2 input framerate. Am I doing this incorrectly or are my expectations incorrect (if so why)?
The code I added is below:
// configure and attach external parameters
mfxStatus sts = AllocAndInitVppDoUse();
MSDK_CHECK_RESULT(sts, MFX_ERR_NONE, sts);
m_FrameRateConversion.Algorithm = MFX_FRCALGM_DISTRIBUTED_TIMESTAMP;
m_mfxVppParams.ExtParam = &m_VppExtParams; // vector is stored linearly in memory
m_mfxVppParams.NumExtParam = (mfxU16)m_VppExtParams.size();
m_VppDoUse.NumAlg = 1;
m_VppDoUse.AlgList = new mfxU32 [m_VppDoUse.NumAlg];
m_VppDoUse.AlgList = MFX_EXTBUFF_VPP_FRAME_RATE_CONVERSION; // turn on FRC
It looks like there may be an issue with using FRC with this algorithm enabled. We are exploring this further to find the root cause.
Are you encountering repeated MFX_ERR_MORE_DATA return code when calling RunFrameVPPAsync()?
As a side note. You can still do FRC but without timestamp interpolation. Just skip pushing m_VppDoUse and m_FrameRateConversion extended buffers to the VPP extended buffer list. However, observe the behavior of RunFrameVPPAsync() for the case when converting frame rate from from instance 30 to 60 fps. This will effectively duplicate each frame, and for each duplicated frame, RunFrameVPPAsync() will require additional output surface, indicated by the return code MFX_ERR_MORE_SURFACE.
Thanks. I'm actually seeing the same behavior (frames in = frames out) no matter which algorithm is selected. I haven't tried increasing the framerate, just stepping down from 60->30 and AFAIK there's no explicit error being generated.
Are you saying that I should handle the FRC myself (dropping/repeating frames) in RunFrameVPPAsync()?
So it turns out using FRC with the MFX_FRCALGM_DISTRIBUTED_TIMESTAMP option is working as expected after all.
There is however an undocumented requirement to ensure correct behavior (we will work to improve the description in future release of Media SDK). User must provide TimeStamp (PTS) that are increasing to ensure accurate behavior, if not there may be unwanted sideeffects like the one we both observed where repeated MFX_ERR_MORE_DATA status is returned.
Here is an example for FRC from 10fps to 29.97fps:
inPTS = 0, 9000, 18000, 27000, … (corresponds 10 fps)
outPTS = 0, 3003, 6006, 9009, … (corresponds 29.97 fps).
Also make sure you handle MFX_ERR_MORE_SURFACE for the case of increasing frame rate as described in earlier post.
Let us know if you are still encountering issues.
I'm actually trying to go the other way (reduce 60FPS to 30FPS). The use of MFX_FRCALGM_DISTRIBUTED_TIMESTAMP may have been misleading, when I couldn't get it working with MFX_FRCALGM_PRESERVE_TIMESTAMP I started trying the other options.
My code is setting the PTS (based on what I'm getting from the container), but I'm not seeing any change on the other side. I haven't tried generating my own PTS though.
When converting from 60->30 I expect to see a 50% drop in the number of frames received from the encoder, but that's not happening. If I send in 5000 I get 5000 out.
are you still encountering issues?
When reducing frame rate, note that RunVPPFrameAsync will return MFX_ERR_MORE_DATA indicating that frame(s) are skipped and not passed on to encoder (for instance using a transcode pipeline).
For the case of just dropping frames such as in the case 60 -> 30 fps, you do not have to apply any special settings or extended buffers to VPP. If you look at the Media SDK 2013 sample_multi_transcode code and use a command line such as below, the only thing you have to do to enable 60 -> 30 FRC is the following:
"sample_multi_transcode.exe -i::h264 in.264 -o::h264 out.264 -b 500 -f 60 -hw"
In "VPPPreInit" comment out the 3 lines of code that only enable VPP in the case of resolution change or de-interlace
//if ( (m_mfxDecParams.mfx.FrameInfo.CropW != pParams->nDstWidth && pParams->nDstWidth) ||
// (m_mfxDecParams.mfx.FrameInfo.CropH != pParams->nDstHeight && pParams->nDstHeight) ||
2) In "InitVppMfxParams" make sure to set output frame rate to 30 such as:
m_mfxVppParams.vpp.Out.FrameRateExtN /= 2;
I had already modified VPPPreInit to not disable VPP in those cases (by detecting if FRC is necessary earlier). It appears that this code block (from InitDecMfxParams):
// if frame rate specified by user set it for decoder and the whole pipeline
ConvertFrameRate(pInParams->dFrameRate, &m_mfxDecParams.mfx.FrameInfo.FrameRateExtN, &m_mfxDecParams.mfx.FrameInfo.FrameRateExtD);
was making it so VPP didn't think it needed to drop frames (I had borrowed the -f argument). After commenting it out I am now seeing the behavior I expected.
Thanks again for the assistance.
I'm using the one of the transcode tutorials and trying to change the video frame rate from 60 to 30 fps.
at some point, before all the input frames are decoded, I'm getting repeated MFX_ERR_MORE_DATA status. is it still a known problem? how can I convert the frame rate successfully? I'm ok with a simple algorithm that is dropping every 2nd input frame...
You can refer to the frame rate conversion article - https://software.intel.com/en-us/articles/frame-rate-conversion-frc-in-intel-media-sdk to decrease the frame rate from 60 to 30fps. Here it is explained with the help of the tutorials.
You can check more technical articles over here.