Linux, Media SDK 2014, h264, low latency encoding

[Nikita_O_]

Hi!

My task is research capabilities of Media SDK hardware encoding with low latency for video conf. I am using core i7 Haswell.

I am encoding video with resolution 3840x1080 (Double FullHD) at 25 fps.

I have tried sample_video_conf, but it it not low latency, time for encoding 1 frame is not stable and can be between 30 and 50 (sometimes more than 50) ms.

Maybe I am doing something wrong?

I have seen sources of video conf example. There is two modes

  if (m_initParams.pBrc.get())
    {
        //brc uses target bitrate value, but Mediasdk required CQP mode
        m_initParams.pBrc->Init(&m_mfxEncParams);

        m_mfxEncParams.mfx.RateControlMethod   = MFX_RATECONTROL_CQP;
        //20 only as a constant, it will be a adjusted by brc
        m_mfxEncParams.mfx.QPI                 = 20;
        m_mfxEncParams.mfx.QPP                 = 20;
        m_mfxEncParams.mfx.QPB                 = 20;
    }
    else
    {
        m_mfxEncParams.mfx.TargetKbps          = m_initParams.nTargetKbps; // in Kbps
        m_mfxEncParams.mfx.RateControlMethod   = MFX_RATECONTROL_VBR; //dynamic bitrate change requires for encoder to be initialized with VBR
    }

If we don`t set bitrate than sample uses MFX_RATECONTROL_CQP, if we set bitrate than sample will use  MFX_RATECONTROL_VBR.

But if I set bitrate for less than 1 Mbit per sec, resulting bitrate is not lower than 1 Mbit. Why?

What about  MFX_RATECONTROL_VCM? Documentation says that this is special mode for videconf, why sample doesn`t use it? And where to find any examples or just any info about usage of  MFX_RATECONTROL_VCM?

[Jeffrey_M_Intel1]

You may be interested in the simpler starting point available in the Media SDK tutorial on the Media Solutions Portal.

This has 2 low latency starting points: simple_6_encode_vmem_lowlatency and simple_6_transcode_opaque_lowlatency

I'm also seeing latencies in the same range as you mentioned with these implementations.  They are very similar on Linux and Windows.  I'll check if there is anything that might be done to optimize latency further. In the meantime, what range of latencies are you looking for?

For BRC, the MaxKbps setting is often not enforced.  Bitrate will vary, often significantly, around the selected rate.  The variance can be reduced by shrinking the BufferSizeInKB parameter. (Current default size is 2 seconds at target bitrate.)  

 MFX_RATECONTROL_VCM isn't implemented for Linux and there is no documentation for it yet.    

[Nikita_O_]

Hi!

Thanks! I will see that samples.

I have 25 fps video and I want to stream it via internet to another point with total latency not more 250ms.

So, it is video conf. task.

Now I have average 40ms latency at encoding stage. As I have 25 fps, 40ms per frame is max. delay. Of cource I can add some output buffer for example for 2 or 3 frames (80-120ms) and that I will hope that using this buffer I will have correct streaming with constant delay 40ms for each frame. But of cource, it will be better to reduce frame encoding latency. So, now I am looking for different solutions.

By the way, As Media SDK has some limitations for system (Ubuntu/SuSE, a short list of kernel versions), libva (vaapi) doesn`t have such limitations. Could you compare Media SDK and VAAPI (libva) for lowlatency case?

[Jeffrey_M_Intel1]

Regardless of any latency optimizations we can find, I suspect there will always be some variation -- especially if you are doing multiple encodes. Buffering is of course not a full solution but will provide more assurance that your application can maintain a constant frame rate.  Hopefully this will be for occasional spikes and not latencies above your target every few frames.

We don't have any latency comparisons vs. a pure libva implementation, and it may take a while to get them.  If you have any comparison data  (vs. libva or a reference SW implementation) you've already generated this could speed up the investigation.  Windows and Linux latencies are almost identical.  In Windows, decode wraps DXVA2 and in Linux decode wraps VAAPI so large differences are not expected.