Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Sync loss between 2 VPP components

Pavel_P_1
Beginner
553 Views

Hello,

I'm trying to make composition with video and ARGB picture on 2017r2 and have the problem.

This problem was described in other topic

https://software.intel.com/en-us/forums/intel-media-sdk/topic/562583

but there is no answer.

My target sheme of execution is:

(session 1: [decoder]) ---> (session 2: [VPP]) ---> (session 3: [composite VPP] ---> [encoder])
                                                                                (ARGB surface) ---^
I use separate session for [VPP] because there can be multiple variants of resolution and frame rate at the same time.

The result of encoding has the next glitches: flipping forward and back between frames.

I made some experiments with different variants of execution scheme (of course all session are connected), and here are the results:

scheme A (works fine):
(session 1: [decoder]) ---> (session 2: [composite VPP] ---> [encoder])
                                           (ARGB surface) ---^

scheme B (works fine):
(session 1: [decoder]) ---> (session 2: [composite VPP]) ---> (session 3: [encoder])
                                            (ARGB surface) ---^
                            
scheme C (works fine):
(session 1: [decoder]) ---> (session 2: [composite VPP]) ---> (session 3: [VPP] ---> [encoder])
                                            (ARGB surface) ---^
                            
scheme D (works with glitches):
(session 1: [decoder]) ---> (session 2: [composite VPP]) ---> (session 3: [VPP]) ---> (session 4: [encoder])
                                            (ARGB surface) ---^
                            
scheme E (works with glitches):
(session 1: [decoder] ---> [VPP]) ---> (session 2: [composite VPP] ---> [encoder])
                                                            (ARGB surface) ---^
                
All the next experiments I made are only with scheme E.

Video - timer, picture - png with dices.

Firstly, I added 250 surfaces to frame pool of [VPP]. Additionally I numbered empty surfaces using vertical light lines (number of light lines is the index of the surface) and received the following result:

https://www.youtube.com/watch?v=Wpuhqh-E_F0

It shows me that [composite VPP] uses surfaces that were taken from [VPP] but not filled with data yet. I.e. [composite VPP] can draw surfaces i and i-1 while [VPP] fills only surface i-1, and i-th is empty (or old).

So I added MFXVideoCORE_SyncOperation() in [VPP] and the glitches are gone. Because it's a rather slow solution, I'm searching for an alternative approach. I changed the direction of drawing surfaces (first - ARGB picture, second - frame from [VPP]) and glitches were gone again (without MFXVideoCORE_SyncOperation()). Next I made the following scheme: I draw the video frame, then ARGB picture, and third - invisible video frame from [VPP] again, and there were no glitches too (without MFXVideoCORE_SyncOperation()). But it's a slow solution too.

Right now i use frame buffer between (VPP) and (composite VPP) nodes. It's a queue with length = sync depth + 1. It solves the problem, but i think, we must find and fix the real error.

What else can I do to find a correct solution of my problem?

0 Kudos
6 Replies
Dmitry_E_Intel
Employee
553 Views

Hello Pavel,

Do you join the sessions by JoinSession() call?

Regards,

Dmitry

0 Kudos
Pavel_P_1
Beginner
553 Views

Hello Dmitry.

Yes, i made connection between sessions by MFXJoinSession() function for all schemes, i tested.

Pavel.

0 Kudos
Dmitry_E_Intel
Employee
553 Views

Did it help?

0 Kudos
Pavel_P_1
Beginner
552 Views

No, it gives no result. The glitches are still here. And in my previous comment, I tried to say, that I used MFX Join Session() in my schemes from the beginning.

0 Kudos
Dmitry_E_Intel
Employee
553 Views

Is it possible to share a reproducer? 

0 Kudos
Molinder__Herman
Beginner
553 Views

Hello,

I've encountered this issue as well. I'm using "scheme E" as Pavel P describes it, with one exception. I'm rendering the [composite VPP] output directly to HDMI, thus no encoding.

Any updates on this issue? Any suggested work arounds?

BR
Herman

0 Kudos
Reply