topic Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit. in Intel® Distribution of OpenVINO™ Toolkit

C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

yanny — Thu, 26 Sep 2024 23:02:38 GMT

Hi,

I have a simple program for matrix multiplication using OpenVINO (Version: 2024.4.0) on Windows 11. It works fine on CPU and GPU, but it crashes on NPU upon exit. If anyone has any clues on how to fix this, I would really appreciate it. Thank you!

Regards,

-yanny

#include <openvino/openvino.hpp> #include <openvino/op/matmul.hpp> #include <iostream> int main() { try { ov::Core core; ov::Shape input_shape_A{ 1, 1024 }; ov::Shape input_shape_B{ 1024, 1024 }; auto input_A = std::make_shared<ov::op::v0::Parameter>(ov::element::f32, input_shape_A); auto input_B = std::make_shared<ov::op::v0::Parameter>(ov::element::f32, input_shape_B); bool transpose_A = false; bool transpose_B = true; auto matmul_op = std::make_shared<ov::op::v0::MatMul>(input_A, input_B, transpose_A, transpose_B); auto result = std::make_shared<ov::op::v0::Result>(matmul_op); ov::ParameterVector inputs{ input_A, input_B }; ov::ResultVector results{ result }; auto model = std::make_shared<ov::Model>(results, inputs, "MatMulModel"); // This works on CPU and GPU auto compiled_model = core.compile_model(model, "NPU"); auto infer_request = compiled_model.create_infer_request(); std::vector<float> input_data_A(1 * 1024, 2.0f); std::vector<float> input_data_B(1024 * 1024, 1.0f); infer_request.set_tensor(input_A, ov::Tensor(ov::element::f32, input_shape_A, input_data_A.data())); infer_request.set_tensor(input_B, ov::Tensor(ov::element::f32, input_shape_B, input_data_B.data())); infer_request.infer(); infer_request.wait(); auto output_tensor = infer_request.get_output_tensor(); const float* output_data = output_tensor.data<float>(); std::cout << "MatMul result: "; for (size_t i = 0; i < 1024; ++i) { std::cout << output_data[i] << " "; } std::cout << std::endl; } catch (std::exception e) { std::cout << "Exception: " << e.what() << std::endl; } return 0; }

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

Wan_Intel — Mon, 30 Sep 2024 12:02:39 GMT

Hi yanny,

Thanks for reaching out to us.

I've built your application in Release mode and run the application with OpenVINO™ Toolkit 2024.4 on a Ubuntu 22.04 LTS machine. The application ran on the NPU plugin successfully and the output results are similar to your output results as shown as follows:

However, I didn't receive the call stack messages. Could you please take a screenshot and share the screenshot with us for further investigation?

Regards,

Wan

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

yanny — Mon, 30 Sep 2024 17:53:43 GMT

Hi Wan,

Thank you so much for testing out the code for me. I am delighted to know that the code works on a different machine.

Here is the callstack and output on my NPU crash. Please let me know if you need more information, I am happy to provide more information.

Exception thrown at 0x00007FF9FBAFFABC in intel-openvino-sample.exe: Microsoft C++ exception: std::runtime_error at memory location 0x000000A9AF6FEB70. 'intel-openvino-sample.exe' (Win32): Loaded 'C:\Program Files (x86)\Common Files\intel\Shared Libraries\bin\tbbbind_2_5.dll'. The thread 23256 has exited with code 0 (0x0). 'intel-openvino-sample.exe' (Win32): Loaded 'C:\Windows\System32\cryptbase.dll'. Symbols loaded without source information. The thread 32368 has exited with code 0 (0x0). The thread 31828 has exited with code 0 (0x0). npu_level_zero_umd.dll!00007ff955607a4b() Unknown npu_level_zero_umd.dll!00007ff95558cfb5() Unknown npu_level_zero_umd.dll!00007ff955576a3a() Unknown npu_level_zero_umd.dll!00007ff9555f3193() Unknown npu_level_zero_umd.dll!00007ff9555f2f41() Unknown npu_level_zero_umd.dll!00007ff9555f326d() Unknown npu_level_zero_umd.dll!00007ff9555f2457() Unknown npu_level_zero_umd.dll!00007ff9555f23d5() Unknown npu_level_zero_umd.dll!00007ff9555f2557() Unknown npu_level_zero_umd.dll!00007ff9555e15a3() Unknown npu_level_zero_umd.dll!00007ff9555e1a1d() Unknown npu_level_zero_umd.dll!00007ff9555e1b42() Unknown ntdll.dll!LdrpCallInitRoutine() Unknown ntdll.dll!LdrShutdownProcess() Unknown ntdll.dll!RtlExitUserProcess() Unknown kernel32.dll!ExitProcessImplementation() Unknown ucrtbase.dll!exit_or_terminate_process() Unknown ucrtbase.dll!common_exit() Unknown > intel-openvino-sample.exe!__scrt_common_main_seh() Line 295 C++ kernel32.dll!BaseThreadInitThunk() Unknown ntdll.dll!RtlUserThreadStart() Unknown

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

Wan_Intel — Tue, 01 Oct 2024 02:28:56 GMT

Hi yanny,

Thanks for the information.

Let me check with relevant team and we'll provide an update as soon as possible.

On the other hand, could you please share the following information with us?

Hardware Specification
Version of Intel® NPU Driver

Regards,

Wan

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

yanny — Tue, 01 Oct 2024 18:15:38 GMT

Device name MSI Prestige 16 AI Evo B1MG Processor Intel(R) Core(TM) Ultra 7 155H 3.80 GHz Installed RAM 32.0 GB (31.7 GB usable) System type 64-bit operating system, x64-based processor

Intel(R) AI Boost Driver Version: 32.0.100.2714

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

Wan_Intel — Sun, 06 Oct 2024 23:09:50 GMT

Hi yanny,

Thanks for your patience.

To further investigate the issue, could you please install the OpenVINO™ 2023.3 LTS from an Archive file via the following link and see if you encounter the same issue when running the C++ application with the NPU plugin?

https://storage.openvinotoolkit.org/repositories/openvino/packages/2023.3/windows

Steps to install the OpenVINO™ 2023.3 LTS on Windows from an Archive File is available here:
https://docs.openvino.ai/2023.3/openvino_docs_install_guides_installing_openvino_from_archive_windows.html

Regards,

Wan

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

yanny — Mon, 07 Oct 2024 14:45:40 GMT

Hi Wan,

OpenVINO™ 2023.3 works on my machine. Thank you very much for your help!

Regards,

-yanny

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

Wan_Intel — Tue, 08 Oct 2024 00:55:57 GMT

Hi yanny,

I'm glad to know that the NPU plugin worked on your machine when using OpenVINO™ 2023.3 LTS.

If you need additional information from Intel, please submit a new question as this thread will no longer be monitored.

Regards,

Wan

Re: C++ OpenVino simple matrix multiplication works on CPU and GPU, but crashes on NPU on exit.

Martin_HZK — Wed, 22 Jan 2025 10:08:39 GMT

Hi yanny,

I am currently getting started to write kernels for NPU, I would like to know what OpenVINO version are you curently using and is it helpful for writing kernels?