Issues in dependency resolution with mfxExtVPPComposite and Opaque memory

[Mark_S_1]

I am attempting to composite five inputs together, using the mfxExtVPPComposite structure.  The inputs come from a variety of sources (file, webcam, rtp), and so are not all the same size, or the same color space.  Therefore, I need to run each input through its own vpp, before compositing them together.  The input vpps send data to the compositing vpp using opaque memory.  There is a single opaque memory pool, since the developer guide states "all input surfaces should be described in one mfxExtOpaqueSurfaceAlloc structure".  This works except that the compositing vpp does not appear to be tracking dependencies properly.  The compositing vpp will sometimes finish its task before the input vpp's have finished their color conversion task.  The result is that old frames, whatever stale data that was in the surface passed to the compositing vpp, will be drawn instead of the new frame.

So assume I have vpp_input1, vpp_input2, vpp_input3, and vpp_compositing.  The execution goes like this:

    Push frame to vpp_input1, it returns MFX_ERR_NONE

    The opaque output surface is passed to vpp_compositing that returns MFX_ERR_MORE_DATA

    Push frame to vpp_input2, it returns MFX_ERR_NONE

    The opaque output surface is passed to vpp_compositing that returns MFX_ERR_MORE_DATA

    Push frame to vpp_input3, it returns MFX_ERR_NONE

    The opaque output surface is passed to vpp_compositing that returns MFX_ERR_NONE

The vpp_input3 operation appears to be guaranteed to finish before the vpp_compositing operation, but not vpp_input1 or vpp_input2.  Sometimes the correct data is shown, sometimes its a stale frame.

The developer guide states: "The SDK tracks the dynamic pipeline construction and verifies dependency on input and output parameters to ensure the
execution order of the pipeline function".  Can you supply more details about how the media sdk verifies dependencies, and how I can make the compositor see each of the input vpps as dependencies?

Mark Samuelson

[JesusE_Intel]

Hi Mark_S_1,

I apologize for the delay in our response. Could you please share the files and steps to reproduce?

Also, which version of MediaSDK are you using?

Regards,

Jesus

[Mark_L_Intel1]

Hi Mark,

Sorry for the late response.

I have submitted a request to dev team and will give you update soon.

Mark Liu

[Pamela_H_Intel]

Apologies for the delays. Media SDK functionality has been superseded by oneVPL, FFmpeg and GStreamer.

For more information see the oneVPL Landing Page: https://www.intel.com/content/www/us/en/developer/tools/oneapi/onevpl.html where you will find, among other information:

oneVPL version 2.x is an API that is the successor to Intel® Media SDK. The Intel Media SDK project is no longer active and will not be updated. Packages and downloads are still available on GitHub*. For continued support and access to new features, Intel Media SDK users are encouraged to move to oneVPL as soon as possible.

·       Upgrade from Intel Media SDK to oneVPL

·       Intel Media SDK Discontinuation Notice

·       Download from GitHub

Let us know if you have any questions about oneVPL or how to convert from MediaSDK.