- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
When running the benchmark_app that is included with the OpenVINO python package, I encounter a GPU hang.
- CPU: Intel Atom(R) x6211E Processor @ 1.30GHz
- GPU: Intel(R) UHD Graphics (iGPU)
- OS: Fedora 41
- Kernel: 6.15.9-101.fc41.x86_64
- Python: 3.12.11
- OpenVINO: 2025.4.0
- Compute-runtime: 24.35.30872.32
I ran benchmark_app with the following arguments:
benchmark_app -m rtmdet-tiny-coco.onnx -t 86400 -api sync -hint latency -d GPU
This results in the following error in benchmark_app:
[ ERROR ] Exception from src/inference/src/cpp/infer_request.cpp:224:
Exception from src/plugins/intel_gpu/src/runtime/ocl/ocl_stream.cpp:424:
[GPU] clWaitForEvents failed with -14 code
Traceback (most recent call last):
File "/home/zuser/.venv/lib64/python3.12/site-packages/openvino/tools/benchmark/main.py", line 624, in main
fps, median_latency_ms, avg_latency_ms, min_latency_ms, max_latency_ms, total_duration_sec, iteration = benchmark.main_loop(requests, data_queue, batch_size, args.latency_percentile, pcseq)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zuser/.venv/lib64/python3.12/site-packages/openvino/tools/benchmark/benchmark.py", line 181, in main_loop
times, total_duration_sec, iteration = self.sync_inference(requests[0], data_queue)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/zuser/.venv/lib64/python3.12/site-packages/openvino/tools/benchmark/benchmark.py", line 106, in sync_inference
request.infer()
File "/home/zuser/.venv/lib64/python3.12/site-packages/openvino/_ov_api.py", line 184, in infer
return OVDict(super().infer(_data_dispatch(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Exception from src/inference/src/cpp/infer_request.cpp:224:
Exception from src/plugins/intel_gpu/src/runtime/ocl/ocl_stream.cpp:424:
[GPU] clWaitForEvents failed with -14 codeIn journalctl I can see the following:
Dec 02 13:30:56 gtx35510d kernel: i915 0000:00:02.0: [drm] Resetting rcs0 for preemption time out
Dec 02 13:30:56 gtx35510d kernel: i915 0000:00:02.0: [drm] benchmark_app[2131852] context reset due to GPU hang
Dec 02 13:30:56 gtx35510d kernel: i915 0000:00:02.0: [drm] GPU HANG: ecode 11:1:8ed9fff3, in benchmark_app [2131852]The hang can take over 24 hours to happen, but most often will happen before then.
I have attached outputs from the following:
- benchmark_app
- lscpu
- clinfo
- journalctl
- hello_query_device
The website does not allow me to upload my .onnx but you can find it here: https://huggingface.co/bukuroo/RTMDet-ONNX/resolve/main/rtmdet-tiny-coco.onnx?download=true
I was able to reproduce the hang on older versions of Fedora, Kernel, OpenVINO and Compute-runtime.
Please advise.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Caelan,
Thank you for reaching out to OpenVINO™ community.
Referring to your hardware specifications and environments, I noticed that the operating system that you are using is not validated by OpenVINO™.
It’s recommended to use the following system if you want to use GPU plugin:
- Windows 11, 64-bit
- Windows 10, 64-bit
- Ubuntu 24.04 long-term support (LTS), 64-bit
- Ubuntu 22.04 long-term support (LTS), 64-bit
- Ubuntu 20.04 long-term support (LTS), 64-bit
- CentOS 7
- Red Hat Enterprise Linux (RHEL) 8 and 9, 64-bit
For more information, please visit https://docs.openvino.ai/2025/about-openvino/release-notes-openvino/system-requirements.html
Regards,
Wan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Wan,
Sorry for the delay, the hang only occurred after almost 24 hours.
I have reproduced the issue with the following setup:
- OS: Ubuntu 24.04.3 LTS
- Kernel: 6.14.0-36-generic
- Python: 3.12.3
- OpenVINO: 2025.4.0
- Compute-runtime: 23.43.027642
Once again, it was produced using the following call:
benchmark_app -m rtmdet-tiny-coco.onnx -t 86400 -api sync -hint latency -d GPU
I've attached the new output from benchmark_app, clinfo and journalctl.
Thanks,
Caelan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Caelan,
Thank you for sharing the information with us.
We will replicate the issue from our end, and we will provide an update here shortly.
Regards,
Wan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Caelan,
Thank you for your patience.
Referring to your previous reply, I noticed you are using Ubuntu 24.04 LTS with Compute Runtime version 23.43.027642.
For your information, Intel Atom® x6211E Processor (Elkhart Lake – Legacy platform) has been validated on the following environment:
- Compute-Runtime version 24.35.30872.22
- Ubuntu 22.04 LTS with stock kernel
You may run the benchmark application with the environment above and see if the issue can be resolved. For more information, please refer to Support for legacy platforms.
Regards,
Wan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Wan,
I was able to reproduce the issue on Ubuntu 22.04 LTS with Compute-Runtime 24.35.30872.22.
Thanks,
Caelan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Caelan,
Thank you for sharing the information.
We are checking on this, and we will provide an update here shortly.
Regards,
Wan
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page