Variable framerate encoding?

[Matej_K_]

I need to encode variable frame rate video, but it looks like the fps fields in FrameInfo are used for ratecontrol, even though the input has perfectly valid timestamps. So how does one encode VFR video using the Media SDK?

[Sravanthi_K_Intel]

Hello there - Can you kindly rephrase your question? We do not completely understand what you're looking for here. Did you mean transcode instead of encode? And for VFR, can't you use timestamps to get the desired output?

In any case, please use the following format to send in details - it helps us immensely - https://software.intel.com/en-us/forums/topic/531083

[Matej_K_]

Hi,

thank you for the response. I am actually doing transcode, but it's the encoder part I have problem with in this instance. The source video has variable framerate - parts are 24000/1001 (telecine converted to 24fps, with TFF/RFF flags stripped) and parts are 60000/1001 (deinterlaced broadcast). 

I can of course use timestamps to get the proper presentation time, but that is not the problem. The problem is ratecontrol. It looks like the encoder uses FPS when determining how many bits each encoded frame gets, instead of frame timestamps. So when FPS is set to 60000/1001 but there are only 24 frames per second, the resulting video bitrate is less than half of what it should be. 

This happens with MFX_RATECONTROL_AVBR, but I think all bitrate based ratecontrols work the same in this regard. Also the encoder will not even initialize when framerate is not specified. x264 for instance does not have same problem. it can work without knowing the framerate, as long as there are proper timestamps (increasing encoding latency by one frame of course). 

Also I am aware that I can change the framerate during encoding using MFXVideoENCODE_Reset, but I'd much prefer not to, as it's just a workaround for something that isn't a problem for all the other encoders that we're using.

[Matej_K_]

Hey,

don't want to be too pushy, but if possible, I'd really like to get confirmation that there is no way to have encoder ratecontrol take timestamps into account instead of relying on framerate.

[Sravanthi_K_Intel]

Hello there - Short answer - We do not cover encoder rate control using timestamps. We use, as you observe, framerate for it. If you would like to use timestamps for it instead, you have to develop your algorithm for it and test it. You could do that using CQP rate control method to do what you are looking for. From the manual (mediasdk-man.pdf: p 183)

Alternatively, the application may use the CQP (constant quantization parameter) encoding 
mode to perform customized bitrate adjustment on a per-frame base. The application may use any of the encoded or display order modes to use per-frame CQP.