Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.

Reusing MFX session after calling EncodeFrameAsync with NULL surface

Elwood_H_
Beginner
349 Views

Hi,

We're building an application where we would like to use the Intel Quick Sync hardware encoder to encode a discontinuous stream: e.g., we have a video stream for 30 seconds, it drops for 10 seconds, it comes back for 30 seconds, etc...

When the stream drops after 30 seconds, it's likely that there are still frames being encoded so what we would like to do is, once we get an external signal that the stream is gone, flush all frames in the encoder's pipelines. As we understood from the documentation and sample_encode, this can be done by calling EncodeFrameAsync with a NULL surface.

The question now is, when the stream comes back, can we still use the same MFXSession?

We modified the sample_encode source code slightly in such a way that in the main() procedure, pPipeline->Run() is called 3 times in a row. When running it the third time, we get a MFX_ERR_LOCK_MEMORY status from the Direct3D allocator when trying to lock a frame in the Run() procedure.

Does this mean that calling EncodeFrameAsync with a NULL surface sets the MFXSession in some kind of "closing" state after which we always have to close it completely and reinitialize it? The odd thing is then that the error apparently originates from the Direct3D device...

Note that this only occurs when using Direct3D surfaces.

Regards,

Elwood

0 Kudos
5 Replies
Surbhi_M_Intel
Employee
349 Views

Hi Elwood, 

The locking & unlocking of surfaces happens only when using d3d surfaces(video memory). S. EncodeFrameAsycn is responsible for taking the input in display or encoded order and it is call with NULL parameter to get buffered frames from encoder. Since it's an asynchronous pipeline make sure you use a sync operation to flush all the frames processed and re-use those surfaces. MFX_ERR_MEMORY_LOCK is indicated the surfaces are locked and in use so you must call sync operation. Also, If the videos are different & have different properties, then the parameters will be reinitialized. 

I haven't tried this case, let me discuss this internally & we will get back to you. 

 

Thanks,
Surbhi

 

0 Kudos
Elwood_H_
Beginner
349 Views

Hi Surbhi,

Thanks for you reply.

From what we've understood from the Run() procedure in the CEncodingPipeline class in the sample_encode project, the sync operation is indeed used to synchronously wait until the pipelines are idle again. So we assumed executing the Run() procedure a few consecutive times, would not pose a problem (yet it does).

Basically, we only changed the main() procedure of the sample_encode.cpp to this (so the video parameters are also unaltered)

    ...

    for (;;)
    {
        sts = pPipeline->Run();
        sts = pPipeline->Run();  // this line was added
        sts = pPipeline->Run();  // this line was added

        if (MFX_ERR_DEVICE_LOST == sts || MFX_ERR_DEVICE_FAILED == sts)
        {
            msdk_printf(MSDK_STRING("\nERROR: Hardware device was lost or returned an unexpected error. Recovering...\n"));
            sts = pPipeline->ResetDevice();

    ...

After calling the Run() procedure twice, we didn't get any error, yet the third time we did.

Regards,

Elwood

0 Kudos
Surbhi_M_Intel
Employee
349 Views

Hi Elwood, 

I believe the reason you are calling the Run() func thrice is to resolve this issue - "When the stream drops after 30 seconds, it's likely that there are still frames being encoded so what we would like to do is,, once we get an external signal that the stream is gone, flush all frames in the encoder's pipelines. As we understood from the documentation and sample_encode, this can be done by calling EncodeFrameAsync with a NULL surface." If you see the Run() in pipeline_encode.cpp there is an extra loop to get the buffered frames out, which should take care of the above concern. 
Also, calling Run() again multiple times the application is looking for free surfaces & locking-unlocking them which is waste of resources. 
Did you come across any issue when you were use the Run() once instead multiple times in your application?

Thanks,
Surbhi

0 Kudos
Elwood_H_
Beginner
349 Views

Hi Surbhi,

Yes, we modified the example to reproduce the behavior I described before more easily: the pattern we use in our code for encoding and flushing the buffers when video lost is exactly the same as in the sample. And we're also aware that calling the Run() procedure three times in a row with the same data doesn't make any sense in practice; it's simply a way to reproduce the error (which doesn't occur when only calling the Run() procedure once).

We were simply under the assumption that after calling the Run() procedure, the encoding pipeline returns to its initial state. But apparently this is not 100% correct.

Regards,

Elwood

0 Kudos
Surbhi_M_Intel
Employee
349 Views

Hi Elwood, 

okay,got it. Calling Run() thrice was a reproducer and not the original pipeline. Going back  to your original question - 
The question now is, when the stream comes back, can we still use the same MFXSession? 
Yes you can use the same mfxsession but not the mfxencode. 

Does this mean that calling EncodeFrameAsync with a NULL surface sets the MFXSession in some kind of "closing" state after which we always have to close it completely and reinitialize it?
we have discussed about this earlier, what EncodeFrameAsync and Syncoperation does. It does not closes the encode session or re-initialize it. 

We were simply under the assumption that after calling the Run() procedure, the encoding pipeline returns to its initial state
No, Run()  doesn't put the encoder in the original state. 

The error in your application could be most likely because of not closing the encoder. You need to close the encoder after the first 30seconds video and call init() again for the next video. By that you should see the MEMORY_LOCK_ERR in your application. 
Let us know is that solves the issue. 

 

Thanks,
Surbhi

 

 

0 Kudos
Reply