topic Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa in Graphics

ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Tue, 17 Dec 2024 17:07:52 GMT

ffmpeg 7.1 and Mesa 24.3.1 on Linux (openSUSE Tumbleweed-Slowroll) are expected to support h264_vulkan and hevc_vulkan video encoding (transcoding).

Simple ffmpeg decode benchmarks tests seemingly works OK.

If here are some missing steps like additional export variables or something wrong with the ffmpeg command lines used, I would be happy to hear back and test alternatives?

ls /usr/share/vulkan/icd.d
intel_hasvk_icd.x86_64.json intel_icd.x86_64.json lvp_icd.x86_64.json

export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/intel_icd.x86_64.json
export ANV_VIDEO_DECODE=1
export ANV_VIDEO_ENCODE=1

vulkaninfo | egrep -i 'decode|encode'
VK_KHR_video_decode_h264 : extension revision 9
VK_KHR_video_decode_h265 : extension revision 8
VK_KHR_video_decode_queue : extension revision 8
VK_KHR_video_encode_h264 : extension revision 14
VK_KHR_video_encode_h265 : extension revision 14
VK_KHR_video_encode_queue : extension revision 12
queueFlags = QUEUE_VIDEO_DECODE_BIT_KHR | QUEUE_VIDEO_ENCODE_BIT_KHR
VIDEO_CODEC_OPERATION_ENCODE_H264_BIT_KHR
VIDEO_CODEC_OPERATION_ENCODE_H265_BIT_KHR
VIDEO_CODEC_OPERATION_DECODE_H264_BIT_KHR
VIDEO_CODEC_OPERATION_DECODE_H265_BIT_KHR

vulkaninfo --summary
==========
VULKANINFO
==========

Vulkan Instance Version: 1.3.296

Instance Extensions: count = 24
-------------------------------
VK_EXT_acquire_drm_display : extension revision 1
VK_EXT_acquire_xlib_display : extension revision 1
VK_EXT_debug_report : extension revision 10
VK_EXT_debug_utils : extension revision 2
VK_EXT_direct_mode_display : extension revision 1
VK_EXT_display_surface_counter : extension revision 1
VK_EXT_headless_surface : extension revision 1
VK_EXT_surface_maintenance1 : extension revision 1
VK_EXT_swapchain_colorspace : extension revision 5
VK_KHR_device_group_creation : extension revision 1
VK_KHR_display : extension revision 23
VK_KHR_external_fence_capabilities : extension revision 1
VK_KHR_external_memory_capabilities : extension revision 1
VK_KHR_external_semaphore_capabilities : extension revision 1
VK_KHR_get_display_properties2 : extension revision 1
VK_KHR_get_physical_device_properties2 : extension revision 2
VK_KHR_get_surface_capabilities2 : extension revision 1
VK_KHR_portability_enumeration : extension revision 1
VK_KHR_surface : extension revision 25
VK_KHR_surface_protected_capabilities : extension revision 1
VK_KHR_wayland_surface : extension revision 6
VK_KHR_xcb_surface : extension revision 6
VK_KHR_xlib_surface : extension revision 6
VK_LUNARG_direct_driver_loading : extension revision 1

Instance Layers: count = 3
--------------------------
VK_LAYER_KHRONOS_validation Khronos Validation Layer 1.3.296 version 1
VK_LAYER_MESA_device_select Linux device selection layer 1.3.211 version 1
VK_LAYER_MESA_overlay Mesa Overlay layer 1.3.211 version 1

Devices:
========
GPU0:
apiVersion = 1.3.296
driverVersion = 24.3.1
vendorID = 0x8086
deviceID = 0x56a1
deviceType = PHYSICAL_DEVICE_TYPE_DISCRETE_GPU
deviceName = Intel(R) Arc(tm) A750 Graphics (DG2)
driverID = DRIVER_ID_INTEL_OPEN_SOURCE_MESA
driverName = Intel open-source Mesa driver
driverInfo = Mesa 24.3.1
conformanceVersion = 1.3.6.0
deviceUUID = 8680a156-0800-0000-0300-000000000000
driverUUID = 6aa00ddb-7b3f-481c-b7fa-218b32c5137d

#######################

inxi -CGSz
System:
Kernel: 6.6.65-1-longterm arch: x86_64 bits: 64
Desktop: GNOME v: 47.2 Distro: openSUSE Tumbleweed-Slowroll 20241202
CPU:
Info: 12-core (8-mt/4-st) model: 12th Gen Intel Core i7-12700KF bits: 64
type: MST AMCP cache: L2: 12 MiB
Speed (MHz): avg: 831 min/max: 800/4900:5000:3800 cores: 1: 831 2: 831
3: 831 4: 831 5: 831 6: 831 7: 831 8: 831 9: 831 10: 831 11: 831 12: 831
13: 831 14: 831 15: 831 16: 831 17: 831 18: 831 19: 831 20: 831
Graphics:
Device-1: Intel DG2 [Arc A750] driver: i915 v: kernel
Device-2: Microdia Camera driver: snd-usb-audio,uvcvideo type: USB
Display: x11 server: X.org v: 1.21.1.14 with: Xwayland v: 24.1.4 driver:
X: loaded: modesetting unloaded: vesa dri: iris gpu: i915
resolution: 2560x1440~60Hz
API: EGL v: 1.5 drivers: iris,swrast platforms: gbm,x11,surfaceless,device
API: OpenGL v: 4.6 compat-v: 4.5 vendor: intel mesa v: 24.3.1
renderer: Mesa Intel Arc A750 Graphics (DG2)
API: Vulkan v: 1.3.296 drivers: N/A surfaces: xcb,xlib

====================

ffmpeg version 7.1 Copyright (c) 2000-2024 the FFmpeg developers
built with gcc 14 (SUSE Linux)

ffmpeg -hide_banner -encoders | grep vulkan
V....D h264_vulkan H.264/AVC (Vulkan) (codec h264)
V....D hevc_vulkan H.265/HEVC (Vulkan) (codec hevc)

######################

ffmpeg -hide_banner -init_hw_device vulkan -i hdv09_04.m2t -vf "format=nv12,hwupload" -c:v h264_vulkan -y hdv09_04_h264_vulkan_nv12.mp4

[mpeg2video @ 0x55db4fe22d80] Invalid frame dimensions 0x0.
Last message repeated 3 times
[mpegts @ 0x55db4fe042c0] PES packet size mismatch
[mpegts @ 0x55db4fe042c0] Packet corrupt (stream = 1, dts = 258142320).
[mpegts @ 0x55db4fe042c0] Could not find codec parameters for stream 2 (Unknown: none ([160][0][0][0] / 0x00A0)): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[mpegts @ 0x55db4fe042c0] Could not find codec parameters for stream 3 (Unknown: none ([161][0][0][0] / 0x00A1)): unknown codec
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, mpegts, from 'hdv09_04.m2t':
Duration: 00:03:59.06, start: 2629.496000, bitrate: 26110 kb/s
Program 100
Stream #0:0[0x810]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, top first), 1440x1080 [SAR 4:3 DAR 16:9], 25000 kb/s, 25 fps, 25 tbr, 90k tbn
Side data:
cpb: bitrate max/min/avg: 25000000/0/0 buffer size: 7340032 vbv_delay: N/A
Stream #0:1[0x814]: Audio: mp2 (mp3float) ([3][0][0][0] / 0x0003), 48000 Hz, stereo, fltp, 384 kb/s
Stream #0:2[0x815]: Unknown: none ([160][0][0][0] / 0x00A0)
Stream #0:3[0x811]: Unknown: none ([161][0][0][0] / 0x00A1)
Stream mapping:
Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_vulkan))
Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[h264_vulkan @ 0x55db4fe236c0] No rate control settings specified, using fixed QP = 18
Output #0, mp4, to 'hdv09_04_h264_vulkan_nv12.mp4':
Metadata:
encoder : Lavf61.7.100
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), vulkan(tv, bt709, top coded first (swapped)), 1440x1080 [SAR 4:3 DAR 16:9], q=2-31, 25 fps, 12800 tbn
Metadata:
encoder : Lavc61.19.100 h264_vulkan
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc61.19.100 aac

[h264_vulkan @ 0x55db4fe236c0] Encode failed: -729850096.
[vost#0:0/h264_vulkan @ 0x55db4fe5d480] Error submitting video frame to the encoder
[vost#0:0/h264_vulkan @ 0x55db4fe5d480] Error encoding a frame: Error number -729850096 occurred
[vost#0:0/h264_vulkan @ 0x55db4fe5d480] Task finished with error code: -729850096 (Error number -729850096 occurred)
[vost#0:0/h264_vulkan @ 0x55db4fe5d480] Terminating thread with return code -729850096 (Error number -729850096 occurred)
Unable to submit command buffer: VK_ERROR_DEVICE_LOST
[hwupload @ 0x7ff7c0004440] Failed to upload frame: -542398533.
[vf#0:0 @ 0x55db4fd17100] Error while filtering: Generic error in an external library
[vf#0:0 @ 0x55db4fd17100] Task finished with error code: -542398533 (Generic error in an external library)
[vf#0:0 @ 0x55db4fd17100] Terminating thread with return code -542398533 (Generic error in an external library)
[out#0/mp4 @ 0x55db4fd198c0] video:0KiB audio:19KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 5.515885%
frame= 0 fps=0.0 q=0.0 Lsize= 20KiB time=N/A bitrate=N/A speed=N/A
[aac @ 0x55db4fe069c0] Qavg: 1286.721
Conversion failed!

========================

ffmpeg -hide_banner -init_hw_device vulkan=vkdev:0 -filter_hw_device vkdev -i h264_8bit_yuv420p.mp4 -filter:v "format=nv12,hwupload" -c:v hevc_vulkan hevc_vulkan_8b420.mp4

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'h264_8bit_yuv420p.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf61.1.100
Duration: 00:01:11.20, start: 0.000000, bitrate: 8464 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, top first), 1920x1080 [SAR 1:1 DAR 16:9], 8462 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
File 'hevc_vulkan_8b420.mp4' already exists. Overwrite? [y/N] y
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> hevc (hevc_vulkan))
Press [q] to stop, [?] for help
[swscaler @ 0x7f0aa0014b00] deprecated pixel format used, make sure you did set range correctly
[hevc_vulkan @ 0x5583b890fdc0] No rate control settings specified, using fixed QP = 18
Output #0, mp4, to 'hevc_vulkan_8b420.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf61.7.100
Stream #0:0(und): Video: hevc (Main) (hev1 / 0x31766568), vulkan(pc, smpte170m/unknown/unknown, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 12800 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc61.19.100 hevc_vulkan

[hevc_vulkan @ 0x5583b890fdc0] Encode failed: -1744827296.
[vost#0:0/hevc_vulkan @ 0x5583b890aec0] Error submitting video frame to the encoder
[vost#0:0/hevc_vulkan @ 0x5583b890aec0] Error encoding a frame: Error number -1744827296 occurred
[vost#0:0/hevc_vulkan @ 0x5583b890aec0] Task finished with error code: -1744827296 (Error number -1744827296 occurred)
[vost#0:0/hevc_vulkan @ 0x5583b890aec0] Terminating thread with return code -1744827296 (Error number -1744827296 occurred)
[out#0/mp4 @ 0x5583b883ecc0] video:0KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
[out#0/mp4 @ 0x5583b883ecc0] Output file is empty, nothing was encoded(check -ss / -t / -frames parameters if used)
frame= 0 fps=0.0 q=0.0 Lsize= 0KiB time=N/A bitrate=N/A speed=N/A
Conversion failed!

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Wed, 18 Dec 2024 03:22:29 GMT

Hello terje,

Thank you for reaching out to us regarding the issue you’re experiencing with your system. To better assist you, I need to gather some additional information. Could you please answer the following questions?

Has this issue occurred before?
Can you please run the Intel® System Support Utility for the Linux* Operating System and share the logs with me?
What troubleshooting steps have you tried so far to fix the issue?

Providing this information will help me diagnose the system more accurately and check for available solutions. Thank you for your cooperation, and I look forward to your response.

Best regards,

Randy T.

Intel Customer Support Technician

Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Wed, 18 Dec 2024 10:09:10 GMT

Hi Randy T.

Answers to your questions follows:

1. The issue is new, because ffmpeg Vulkan hevc and h264 video encoding is new in MESA/ANV.

Else I have done several equivalent video accelerated encodings via ffmpeg, vaapi and qsv.

2. Attach the default "localhost.localdomain.txt" log file.

3. I have requested this issue on the ffmpeg-user mailing list. And a similar issue ticket has been reported further as replied to this thread:

https://ffmpeg.org//pipermail/ffmpeg-user/2024-December/058929.html

Thanks,

Terje J. H

Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Wed, 18 Dec 2024 13:45:31 GMT

Add a new ssu log file after installing wodim and route (deprecated package).

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Thu, 19 Dec 2024 00:50:09 GMT

Hello terje,

Thank you for bringing this matter to our attention. We will check on this with our team and review the system information and the complexity of the issue. We will get back to you as soon as we have completed our review.

Best regards,

Randy T.

Intel Customer Support Technician

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Tue, 24 Dec 2024 03:44:47 GMT

Hello terje,

Based on our analysis, the error you encountered might be related to specific input/output format issues or driver-related problems that require further troubleshooting.

Therefore, I would request you to:

Check if there are any known issues with the specific Vulkan driver (Mesa) you are using.
Add the "-loglevel debug" parameter to your current command to gather more detailed error information.

These steps should help us identify the root cause of the issue more effectively.

Best regards,

Randy T.

Intel Customer Support Technician

Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Tue, 24 Dec 2024 14:02:06 GMT

ffmpeg -loglevel debug -init_hw_device vulkan=vkdev:0 -filter_hw_device vkdev -i h264_8bit_yuv420p.mp4 -filter:v "format=nv12,hwupload" -c:v hevc_vulkan hevc_vulkan_8b420.mp4 2>&1 | tee ffmpeg_debug_vulkan_encode.log

Attach the latter output "ffmpeg_debug_vulkan_encode.log" file

Also here is a ffmpeg vulkan decode benchmark working for the same input file according to

https://trac.ffmpeg.org/wiki/HWAccelIntro#Vulkan

ffmpeg -hide_banner -init_hw_device "vulkan=vk:0" -hwaccel vulkan -hwaccel_output_format vulkan -i h264_8bit_yuv420p.mp4 -f null - -benchmark
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'h264_8bit_yuv420p.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf61.1.100
Duration: 00:01:11.20, start: 0.000000, bitrate: 8464 kb/s
Stream #0:0[0x1](und): Video: h264 (High) (avc1 / 0x31637661), yuvj420p(pc, smpte170m/unknown/unknown, top first), 1920x1080 [SAR 1:1 DAR 16:9], 8462 kb/s, 25 fps, 25 tbr, 12800 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> wrapped_avframe (native))
Press [q] to stop, [?] for help
Output #0, null, to 'pipe:':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf61.7.100
Stream #0:0(und): Video: wrapped_avframe, vulkan(pc, smpte170m/unknown/unknown, top coded first (swapped)), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 25 fps, 25 tbn (default)
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
encoder : Lavc61.19.100 wrapped_avframe
[out#0/null @ 0x5616c738ce80] video:765KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: unknown
frame= 1780 fps=918 q=-0.0 Lsize=N/A time=00:01:11.20 bitrate=N/A speed=36.7x
bench: utime=0.588s stime=0.493s rtime=1.940s
bench: maxrss=183376KiB

Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Sat, 28 Dec 2024 11:41:08 GMT

As here is no real reply or suggestion to try yet, I also attach a full, current "vulkaninfo.log" in case it may be of some help to your team?

And another new post of interest with regards to Vulkan FFV1 encoding, beside AV1:

[FFmpeg-user] ffv1_vulkan issues

https://ffmpeg.org//pipermail/ffmpeg-user/2024-December/058956.html

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Sun, 29 Dec 2024 21:18:47 GMT

Hello terje,

I don't have an answer at this moment, but I'll investigate it and get back to you as soon as possible. I'm going to look into the issue further.

Randy T.

Intel Customer Support Technician

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Sun, 05 Jan 2025 20:21:20 GMT

Hello terje,

I wanted to inform you that the encoding with Vulkan implementation must be carried out by software and not the GPU media. Therefore, I suggest continuing to report this issue to the FFmpeg developer who committed the Vulkan function into FFmpeg.

Additionally, could you please verify if the same issue occurs on Ubuntu 23.04 or 22.04?

Randy T.

Intel Customer Support Technician

Re: ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Sun, 05 Jan 2025 23:16:50 GMT

Hello Randy T.,

Thank you for your reply. But I have to get verified if not Intel's ANV driver in Mesa has gpu accelerated Vulkan video encoding capability (similar like vaapi and qsv)?

Reference articles:

https://www.phoronix.com/news/Intel-Vulkan-Video-H264-H265

https://www.phoronix.com/news/Mesa-24.3-Released

Thanks,

Terje J. Hanssen

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Tue, 07 Jan 2025 20:52:00 GMT

Hello terje,

Regarding your question, I need to verify if Intel's ANV driver in Mesa has GPU-accelerated Vulkan video encoding capability, similar to VAAPI and QSV. Let me cross-check this source before providing you with accurate information. I appreciate your patience.

Randy T.

Intel Customer Support Technician

Re: Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

terje — Wed, 22 Jan 2025 21:11:55 GMT

Hello Randy T.,

Any clarification and progress regarding the above mentioned?

Terje J. Hanssen

Re:ffmpeg h264_vulkan and hevc_vulkan Encode failed on A750/Linux/Mesa

RandyT_Intel — Wed, 05 Feb 2025 03:24:14 GMT

Hello terje,

Here's the update.

Graphics hardware decoding, encoding and video processing for Intel Integrated and discrete platforms are supported using the https://github.com/intel/media-driver.

Enabling Media on Vulkan driver is an effort from the community. There is code added a while back in the Vulkan driver to enable HEVC and H.264 for older platforms but it hasn't been thoroughly tested on Alchemist and Battlemage. By default support is disabled unless you set the following environment variable : ANV_VIDEO_DECODE

Having said that, please reach out to ffmpeg forums or the Linux community support to validate if vulkan encoding on Linux is achievable on current slate of drivers.

Randy T.

Intel Customer Support Technician