Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

QuickSync get stuck after 2 reconfigurations

Vladimir_H_
Beginner
548 Views

We have some strange issue with QuickSync encoder. During normal encoding we doing 2 reconfigurations (MFXVideoENCODE_Reset). After that, encoder stucks in following state:
1. All sync points completed (no active sync points).
2. 6 frames locked by quicksync, i.e. Locked = 1.
3. EncodeFrameAsync returns MFX_WRN_DEVICE_BUSY warning while trying to submit next frame or flush.
So in this state I can not submit next frame to encoder and there is nothing to read from encoder (bitstream).
 
We added detection for this case and restart encoder when this situation detected. And this works fine when I submit frames in video memory. But in case when buffers in system memory submitted, encoding ends in 8 locked frames and waiting in memory allocator for first available buffer to submit 9th frame. So app freezes in this state.
We using Intel Media SDK 2016 R2 (7.0.0.358). I have dump already with mentioned freeze and can provide it, and will also provide more info as soon as I get that from test team.

0 Kudos
6 Replies
Mark_L_Intel1
Moderator
548 Views

Hi Vladimir,

As my understanding, there are 2 loops for the encoding process:

loop 1: mfxENC.EncodeFrameAsync(NULL, pENCSurfaces[idx], &mfxBS, &syncp)

loop 2: mfxENC.EncodeFrameAsync(NULL, NULL, &mfxBS, &syncp)

loop 2 is required since there are still uncompleted frame in the surface pool, the purpose of the sync point is to make sure the complete of a single frame but not guarantee the completion of the different frames.

So this process also applied to the encoder reset, could you confirm the "loop 2" also being done in your case?

Mark Liu

0 Kudos
Vladimir_H_
Beginner
548 Views

Hi Mark,

Is there some way to change settings on the fly similar to x264_encoder_reconfig?

How long this will take to get remaining buffers from QS encoder before I can call Reset? (App must stop to submit frames during that time)

0 Kudos
Mark_L_Intel1
Moderator
548 Views

Hi Vladimir,

If my suggestion is right, the BUSY happens because the buffer was still on the previous content and being lock, I am not sure if there is a force flush operation but once you run the second loop, the buffer should be released for the reset, how long it takes can't be decided in this case.

Are you in a situation of time restricted? If yes, then after we do the test of running the second loop and get rid of the device busy error, we need to find a way to force the buffer unlock.

It looks like you want the reset urgent and don't want to stop the app, I am trying to understand:

What I suggested in the flow of my mind:

Set the parameters->MFXVideoENCODE_Init()->initialize buffer->loop#1->loop#2->MFXVideoENCODE_Reset()->go back to loop#1, does the final step means the "App must stop" in your comment?

Mark Liu

0 Kudos
Vladimir_H_
Beginner
548 Views

Hi Mark,

Thank you for clarifications. I understood that I must flush encoder before Reset. I already implemented this in our application and sent for testing. I will measure how long time this takes to flush encoder. Maybe this is quite short time and there no issues at all. Main issue that I worry about is that encoding stops during this flushing+reset. Let me describe example use case. For example, for online video broadcasting, we start encoding with some bitrate. Then at some point in time internet bandwidth drops down. We can detect this and adjust bitrate. Then when we detect that bandwidth is normal again, we can change bitrate back to original.
I think some async version of Reset may help. I.e. already cached frames can be processed with old settings still, and newly submitted frames will be processed with new settings.

0 Kudos
Mark_L_Intel1
Moderator
548 Views

Got it,

In your example, you want to change the bit rate dynamically via the automated bit rate control mode, which we don't recommend. Also the flush will cause serval hundred millisecond which probably doesn't satisfy your application need.

The right way is to do the CQP bit rate control which is described in the developer guide with the following link:

https://software.intel.com/sites/default/files/managed/09/02/Intel_Media_Developers_Guide.pdf

Under section 9.6, there is a detailed information about the customized CQP implementation. Basically in function MFXVideoENCODE_EncodeFrameAsync(), you can pass in the structure mfxEncodeCtrl with the parameter of the target bit rate control. This way, you can change the bit rate when the pipeline is running so you don't have to stop the pipeline.

Let me know if this solves your problem.

Mark Liu

0 Kudos
Vladimir_H_
Beginner
548 Views

Thank you. We will check this.

0 Kudos
Reply