Multiple encode sessions

[GordonTyler]

I'm using OneVPL to encode two AVC streams concurrently. For now, in my test code, I'm using a single thread and calling MFXVideoENCODE_EncodeFrameAsync for each session sequentially.

For example, some pseudo code:

mfxSession session1 = InitEncodeSession();
mfxSession session2 = InitEncodeSession();

while (true) {
  EncodeFrame(session1);
  EncodeFrame(session2);
}

However, MFXVideoENCODE_EncodeFrameAsync is returning MFX_ERR_DEVICE_FAILED (-17) when I call it for the second frame of session2:

EncodeFrame(session1)
EncodeFrame(session2)
EncodeFrame(session1)
EncodeFrame(session2) -> MFX_ERR_DEVICE_FAILED

If I add a call to MFXVideoENCODE_EncodeFrameAsync with a null surface pointer to encode delayed frames then I don't get the error.

Example pseudo code:

mfxSession session1 = InitEncodeSession();
mfxSession session2 = InitEncodeSession();

while (true) {
  EncodeFrame(session1);
  EncodeDelayedFrames(session1);
  EncodeFrame(session2);
  EncodeDelayedFrames(session2);
}

But obviously this is not good for performance, and would still require a mutex when implemented in a multi-threaded system, which would further impact performance.

Any ideas what the underlying problem is here?

[SreedeviK_Intel]

Hi,

Thank you for posting in Intel communities.

To assist you better, could you please provide the following details:

1. OS and Hardware details

2. oneVPL version

Also, please refer to below link on running multiple oneVPL sessions independently:

https://spec.oneapi.io/versions/1.1-rev-1/elements/oneVPL/source/programming_guide/VPL_prg_session.html#:~:text=the%20system%20enumeration.-,Multiple%20Sessions,-%C2%B6

Regards,

Sreedevi

[GordonTyler]

Windows 10 22H2 (19045) on Dell Precision 5560 (Intel Core i7-11800H).

oneVPL 2023.3.1 compiled from source using CMake and Visual Studio 2022.

One more bit of information: I was setting mfxVideoParam.IOPattern to in/out system memory but when I switched that to video memory, the original example I gave now works (and performs better). However, if I then switch to a dual-threaded implementation, I get the same problem again.

Thanks for the link regarding multiple sessions -- I hadn't seen that. I am using a separate session for each encoder. I'll try joining the sessions to see if that helps.

[GordonTyler]

I tried joining the two encode sessions but it has no effect on the multi-threaded scenario. I still get the same original failure mode.

[SreedeviK_Intel]

Hi,

Thank you for sharing the details.

Glad to know you figured it with switching to video memory. 

To assist you better, could you please provide your sample reproducer code along with the exact steps to perform so that we could try it out from our side.

Regards,

Sreedevi 

[GordonTyler]

My original implementation is a combination of C# and C++/CLR. I wrote a C++/CLR wrapper around oneVPL, which is called by a C# application. However, when I tried to re-implement the multi-threaded scenario in pure C++, I could not reproduce the error. I will continue investigating to determine if my C++ example is not representative or if I have a bug in my C++/CLR wrapper that's causing the issue.

[GordonTyler]

Something that's not clear to me from the API docs is the expected lifetime of the pointer to mfxVideoParam and mfxExtBuffer pointers in its ExtParams field, that is passed to MFXVideoENCODE_Init. Does the library copy those pointers and reference them later on after MFXVideoENCODE_Init has returned?

[GordonTyler]

So it appears that pointers in the mfxEncodeCtrl structure passed to MFXVideoENCODE_EncodeFrameAsync must live beyond the call to MFXVideoENCODE_EncodeFrameAsync. My mfxEncodeCtrl, including all the pointers inside it, was stack allocated, but when I moved it to the heap, the problem no longer occurs.

[SreedeviK_Intel]

Hi,

Thanks for sharing your observations with us.

Can you please confirm if your issue is resolved. If not, as mentioned previously please share your sample reproducer code with the exact steps to perform.

Kindly let us know if we can stop monitoring this thread.

Regards,

Sreedevi

[GordonTyler]

My issue is resolved. Thanks for the assistance.

May I suggest that the oneVPL documentation be updated to include information about the expected lifetime of pointers in the various data structures?

[GordonTyler]

Actually, I have one more question. How much advantage is there to joining sessions for a concurrent encoding scenario like this?

[SreedeviK_Intel]

Hi, 

Thank you for letting us know about your resolution and sharing us suggestions on oneVPL documentation.

We have forwarded it with our development team.

Joining two sessions has the of option to set priorities to sessions, and to also can be treated independently to Encode and Decode them. Resharing the same document, please find more below:

https://spec.oneapi.io/versions/1.1-rev-1/elements/oneVPL/source/programming_guide/VPL_prg_session.html#:~:text=the%20system%20enumeration.-,Multiple%20Sessions,-%C2%B6

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Regards,

Sreedevi