Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

lowest latency encode possible

Sam_B_
Beginner
411 Views

Hi there,

How can I achieve the lowest possible encode latency for 1366x768 or 1920x1080 source content that is generated on the fly at 30-60fps? I have used all the settings I saw in the low latency tutorial:

    // Configuration for low latency
    mfxEncParams.AsyncDepth        = 1;
    mfxEncParams.mfx.GopRefDist = 1;

    mfxExtCodingOption extendedCodingOptions;
    memset(&extendedCodingOptions, 0, sizeof(extendedCodingOptions));
    extendedCodingOptions.Header.BufferId        = MFX_EXTBUFF_CODING_OPTION;
    extendedCodingOptions.Header.BufferSz        = sizeof(extendedCodingOptions);
    extendedCodingOptions.MaxDecFrameBuffering    = 1;
    mfxExtBuffer* extendedBuffers[1];
    extendedBuffers[0]            = (mfxExtBuffer*)&extendedCodingOptions;
    mfxEncParams.ExtParam        = extendedBuffers;
    mfxEncParams.NumExtParam    = 1;
    // ---

Also, there is a vpp color conversion from RGB4 to NV12 before the encode.

Right now, my timing indicates that SyncOperation takes 20ms on average, which seems very slow for the HD Graphics 4000??  Also, using GPA shows that the "Media Performance" is not near full utilization.  The encoded content is needed in real time.  Do you have any suggestions for speeding things up?  

Thanks,
Sam

0 Kudos
2 Replies
Petter_L_Intel
Employee
411 Views

Hi Sam,

Right, 20ms is a bit much. On an Intel  3rd generation Core Processor using recent graphics driver (such as 15.31.3071), you should be getting ~10ms or less (for 1080p) for the pipeline you describe, VPP(RGB4 to NV12) -> H,264 Encode.

The configuration as you stated is enough to achieve low latency. But also make sure to set VPP AsyncDepth = 1.

Other things that will increase latency is if you use system memory surfaces instead of D3D surfaces.

You can easily create a VPP-Encode pipeline by slightly modifying the "simple_6_encode - d3d - vpp_preproc" sample from the Media SDK Tutorial. The only thing you will have to add to that sample is the low latency configuration and latency benchmarking (if you wish) from the "simple_6_encode - d3d - lowlatency" sample.

Regards,
Petter 

0 Kudos
Sam_B_
Beginner
411 Views

Hi Petter,

It turned out that changing AsyncDepth = 2 to AsyncDepth = 1 now gets me latencies below 10ms.  Thanks!  I did not expect this since I was calling Encode and then Sync one right after the other.  I guess there is some buffering/deferred processing that happens in the SDK when AsyncDepth > 1?

Thanks,
Sam

0 Kudos
Reply