Media (Intel® Video Processing Library, Intel Media SDK)
Access community support with transcoding, decoding, and encoding in applications using media tools like Intel® oneAPI Video Processing Library and Intel® Media SDK
Announcements
The Intel Media SDK project is no longer active. For continued support and access to new features, Intel Media SDK users are encouraged to read the transition guide on upgrading from Intel® Media SDK to Intel® Video Processing Library (VPL), and to move to VPL as soon as possible.
For more information, see the VPL website.
3075 Discussions

MFX_ERR_DEVICE_FAILED on av_hwframe_transfer_data (RunFrameVPPAsync followed by SyncOperation)

meiler
Novice
1,974 Views

Hi,

 

I'm trying to encode incoming images from an industrial camera and to store it in an h264 encoded video file. Unfortunately after some time I start to see "Error synchronizing the operation" on the console which is the avcodec behaviour when receiving an error from MFXVideoCORE_SyncOperation.

 

Details:

System: Surface Pro 8, Core i7-1185G7

Driver & OS: Windows 10 Pro, Intel Iris XE Graphics Driver 30.0.101.1994 from 13th of May 2022

Video Stream: 3840* 2160, 8bit already converted to NV12. 

Frameworks: ffmpeg 5.1 build from www.gyan.dev

 

I also made sure to set the child_device_type so that when creating the av_hwcontext, libmfx selects the d3d11va backend instead of d3d9 which is the default. (see changeset: https://patchwork.ffmpeg.org/project/ffmpeg/patch/20210820214810.9733-3-artem.galin@gmail.com/ )

 

The actual error occurs after some time independent of the image resolution. A call to MFXVideoVPP_RunFrameVPPAsync is followed by MFXVideoCORE_SyncOperation which in turn returns MFX_ERR_DEVICE_FAILED. The code itself can be analysed here: https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavutil/hwcontext_qsv.c . The error occurs in the function qsv_transfer_data_to.

 

Any ideas on what might go wrong here?


Thanks in advance!
Best Regards,
Michael

0 Kudos
1 Solution
meiler
Novice
1,929 Views

Hi Alekhya,

 

To answer your question I'm using ffmpeg which is compiled against libmfx, meaning the MediaSDK. I posted in these forums since I traced done the error codes to libmfx and having found very little information online on such cases, it seemed like the right place to ask for help. Fortunately, in spite of what I expected, I seem to have found the issue within the last few hours ;).

 

During some further analysis I noticed  high gpu memory usage and my suspicions are that ffmpeg allocated additional hardware buffers that weren't freed in time which caused this issue.  This is the case if the frame dimension isn't a multiple of 16. The code that might be responsible for this behaviour can be found here:

https://github.com/FFmpeg/FFmpeg/blob/e0723b7e4e22492275d476fcd30d759e1198bc5b/libavutil/hwcontext_qsv.c#L1167

 

I changed my frame allocation strategy. I now make sure the AVFrame linesize entries are always a multiple of 16. Which required some changes in the conversion code (padding consideration when converting the image to yuv420p_nv12). I also reuse the AVFrame instances, instead of creating a new one for every single input image. 

 

Best Regards,
MIchael

View solution in original post

0 Kudos
5 Replies
AlekhyaV_Intel
Moderator
1,940 Views

Hi,


Thank you for posting in Intel Communities. Could you please let us know whether you're utilizing Intel Media SDK/Intel oneVPL in your project?


Regards,

Alekhya


0 Kudos
meiler
Novice
1,930 Views

Hi Alekhya,

 

To answer your question I'm using ffmpeg which is compiled against libmfx, meaning the MediaSDK. I posted in these forums since I traced done the error codes to libmfx and having found very little information online on such cases, it seemed like the right place to ask for help. Fortunately, in spite of what I expected, I seem to have found the issue within the last few hours ;).

 

During some further analysis I noticed  high gpu memory usage and my suspicions are that ffmpeg allocated additional hardware buffers that weren't freed in time which caused this issue.  This is the case if the frame dimension isn't a multiple of 16. The code that might be responsible for this behaviour can be found here:

https://github.com/FFmpeg/FFmpeg/blob/e0723b7e4e22492275d476fcd30d759e1198bc5b/libavutil/hwcontext_qsv.c#L1167

 

I changed my frame allocation strategy. I now make sure the AVFrame linesize entries are always a multiple of 16. Which required some changes in the conversion code (padding consideration when converting the image to yuv420p_nv12). I also reuse the AVFrame instances, instead of creating a new one for every single input image. 

 

Best Regards,
MIchael

0 Kudos
TongWu
Employee
1,675 Views

Hi Michael,

 

To reproduce this issue, could you please provide your input file and the command line?

 

Thanks,

Tong

0 Kudos
meiler
Novice
1,651 Views

Hi Tong,

 

It's resolved. The issue was the missing padding in the AVFrame's I created.

 

Best,

Michael

0 Kudos
AlekhyaV_Intel
Moderator
1,867 Views

Hi Michael,


We are working on your issue. We will get back to you soon with an update.


Regards,

Alekhya


0 Kudos
Reply