MFX_ERR_DEVICE_FAILED on av_hwframe_transfer_data (RunFrameVPPAsync followed by SyncOperation)

[meiler]

Hi,

I'm trying to encode incoming images from an industrial camera and to store it in an h264 encoded video file. Unfortunately after some time I start to see "Error synchronizing the operation" on the console which is the avcodec behaviour when receiving an error from MFXVideoCORE_SyncOperation.

Details:

System: Surface Pro 8, Core i7-1185G7

Driver & OS: Windows 10 Pro, Intel Iris XE Graphics Driver 30.0.101.1994 from 13th of May 2022

Video Stream: 3840* 2160, 8bit already converted to NV12. 

Frameworks: ffmpeg 5.1 build from www.gyan.dev

I also made sure to set the child_device_type so that when creating the av_hwcontext, libmfx selects the d3d11va backend instead of d3d9 which is the default. (see changeset: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210820214810.9733-3-artem.galin@gmail.com/ )

The actual error occurs after some time independent of the image resolution. A call to MFXVideoVPP_RunFrameVPPAsync is followed by MFXVideoCORE_SyncOperation which in turn returns MFX_ERR_DEVICE_FAILED. The code itself can be analysed here: https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavutil/hwcontext_qsv.c . The error occurs in the function qsv_transfer_data_to.

Any ideas on what might go wrong here?

Thanks in advance!
Best Regards,
Michael

[AlekhyaV_Intel]

Hi,

Thank you for posting in Intel Communities. Could you please let us know whether you're utilizing Intel Media SDK/Intel oneVPL in your project?

Regards,

Alekhya

[meiler · Accepted solution]

Hi Alekhya,

To answer your question I'm using ffmpeg which is compiled against libmfx, meaning the MediaSDK. I posted in these forums since I traced done the error codes to libmfx and having found very little information online on such cases, it seemed like the right place to ask for help. Fortunately, in spite of what I expected, I seem to have found the issue within the last few hours ;).

During some further analysis I noticed  high gpu memory usage and my suspicions are that ffmpeg allocated additional hardware buffers that weren't freed in time which caused this issue.  This is the case if the frame dimension isn't a multiple of 16. The code that might be responsible for this behaviour can be found here:

https://github.com/FFmpeg/FFmpeg/blob/e0723b7e4e22492275d476fcd30d759e1198bc5b/libavutil/hwcontext_qsv.c#L1167

I changed my frame allocation strategy. I now make sure the AVFrame linesize entries are always a multiple of 16. Which required some changes in the conversion code (padding consideration when converting the image to yuv420p_nv12). I also reuse the AVFrame instances, instead of creating a new one for every single input image. 

Best Regards,
MIchael

[TongWu]

Hi Michael,

To reproduce this issue, could you please provide your input file and the command line?

Thanks,

Tong

[meiler]

Hi Tong,

It's resolved. The issue was the missing padding in the AVFrame's I created.

Best,

Michael

[AlekhyaV_Intel]

Hi Michael,

We are working on your issue. We will get back to you soon with an update.

Regards,

Alekhya