NPU not detected on Ubuntu with OpenVINO 2024.1

[Matthew_C_Intel]

Using a Meteor Lake based NUC (https://www.asus.com/us/displays-desktops/nucs/nuc-mini-pcs/asus-nuc-14-pro/) and OpenVINO 2024.1, but there is no NPU detected in OpenVINO using any app like 'hello_query_device'

Reproduced failure with both Ubuntu 22.04 (Kernel 6.5) and Ubuntu 24.04 (Kernel 6.8)

Steps to reproduce:

$ sudo /bin/bash
# apt update
# apt install --no-install-recommends -q -y software-properties-common ca-certificates gnupg wget build-essential cmake clinfo python3 python3-pip ocl-icd-libopencl1

# wget -qO - https://repositories.intel.com/gpu/intel-graphics.key | gpg --dearmor --output /usr/share/keyrings/intel-graphics.gpg
# echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-graphics.gpg] https://repositories.intel.com/gpu/ubuntu jammy client" | tee /etc/apt/sources.list.d/intel-gpu-jammy.list
# wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16695.4/intel-igc-core_1.0.16695.4_amd64.deb
# wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16695.4/intel-igc-opencl_1.0.16695.4_amd64.deb
# wget https://github.com/intel/compute-runtime/releases/download/24.17.29377.6/intel-level-zero-gpu_1.3.29377.6_amd64.deb
# wget https://github.com/intel/compute-runtime/releases/download/24.17.29377.6/intel-opencl-icd_24.17.29377.6_amd64.deb
# wget https://github.com/intel/compute-runtime/releases/download/24.17.29377.6/libigdgmm12_22.3.19_amd64.deb
# dpkg -i *.deb
# apt update
# apt install --no-install-recommends -q -y level-zero

# wget https://apt.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
# apt-key add GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
# echo "deb https://apt.repos.intel.com/openvino/2024 ubuntu22 main" | tee /etc/apt/sources.list.d/intel-openvino-2024.list
# apt update
# apt install --no-install-recommends -q -y openvino-2024.1.0

# dmesg | grep vpu
[    9.013778] intel_vpu 0000:00:0b.0: enabling device (0000 -> 0002)
[    9.266813] intel_vpu 0000:00:0b.0: [drm] Firmware: intel/vpu/vpu_37xx_v0.0.bin, version: 20230726*MTL_CLIENT_SILICON-release*2101*ci_tag_mtl_pv_vpu_rc_20230726_2101*648a666b8b9
[    9.346605] [drm] Initialized intel_vpu 1.0.0 20230117 for 0000:00:0b.0 on minor 0
# ls /dev/accel/accel0
/dev/accel/accel0

# /usr/share/openvino/samples/cpp/build_samples.sh
# cd /root/openvino_cpp_samples_build/intel64/Release/
# ./hello_query_device

[ INFO ] Build ................................. 2024.1.0-15008-f4afc983258-releases/2024/1
[ INFO ]
[ INFO ] Available devices:
[ INFO ] CPU
[ INFO ]         SUPPORTED_PROPERTIES:
[ INFO ]                 Immutable: AVAILABLE_DEVICES : ""
[ INFO ]                 Immutable: RANGE_FOR_ASYNC_INFER_REQUESTS : 1 1 1
[ INFO ]                 Immutable: RANGE_FOR_STREAMS : 1 18
[ INFO ]                 Immutable: EXECUTION_DEVICES : CPU
[ INFO ]                 Immutable: FULL_DEVICE_NAME : Intel(R) Core(TM) Ultra 5 125H
[ INFO ]                 Immutable: OPTIMIZATION_CAPABILITIES : FP32 FP16 INT8 BIN EXPORT_IMPORT
[ INFO ]                 Immutable: DEVICE_TYPE : integrated
[ INFO ]                 Immutable: DEVICE_ARCHITECTURE : intel64
[ INFO ]                 Mutable: NUM_STREAMS : 1
[ INFO ]                 Mutable: AFFINITY : CORE
[ INFO ]                 Mutable: INFERENCE_NUM_THREADS : 0
[ INFO ]                 Mutable: PERF_COUNT : NO
[ INFO ]                 Mutable: INFERENCE_PRECISION_HINT : f32
[ INFO ]                 Mutable: PERFORMANCE_HINT : LATENCY
[ INFO ]                 Mutable: EXECUTION_MODE_HINT : PERFORMANCE
[ INFO ]                 Mutable: PERFORMANCE_HINT_NUM_REQUESTS : 0
[ INFO ]                 Mutable: ENABLE_CPU_PINNING : YES
[ INFO ]                 Mutable: SCHEDULING_CORE_TYPE : ANY_CORE
[ INFO ]                 Mutable: MODEL_DISTRIBUTION_POLICY : ""
[ INFO ]                 Mutable: ENABLE_HYPER_THREADING : YES
[ INFO ]                 Mutable: DEVICE_ID : ""
[ INFO ]                 Mutable: CPU_DENORMALS_OPTIMIZATION : NO
[ INFO ]                 Mutable: LOG_LEVEL : LOG_NONE
[ INFO ]                 Mutable: CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE : 1
[ INFO ]                 Mutable: DYNAMIC_QUANTIZATION_GROUP_SIZE : 0
[ INFO ]                 Mutable: KV_CACHE_PRECISION : f16
[ INFO ]
[ INFO ] GPU
[ INFO ]         SUPPORTED_PROPERTIES:
[ INFO ]                 Immutable: AVAILABLE_DEVICES : 0
[ INFO ]                 Immutable: RANGE_FOR_ASYNC_INFER_REQUESTS : 1 2 1
[ INFO ]                 Immutable: RANGE_FOR_STREAMS : 1 2
[ INFO ]                 Immutable: OPTIMAL_BATCH_SIZE : 1
[ INFO ]                 Immutable: MAX_BATCH_SIZE : 1
[ INFO ]                 Immutable: DEVICE_ARCHITECTURE : GPU: vendor=0x8086 arch=v785.192.4
[ INFO ]                 Immutable: FULL_DEVICE_NAME : Intel(R) Arc(TM) Graphics (iGPU)
[ INFO ]                 Immutable: DEVICE_UUID : 8680557d080000000002000000000000
[ INFO ]                 Immutable: DEVICE_LUID : 409a0000499a0000
[ INFO ]                 Immutable: DEVICE_TYPE : integrated
[ INFO ]                 Immutable: DEVICE_GOPS : {f16:7884.8,f32:3942.4,i8:15769.6,u8:15769.6}
[ INFO ]                 Immutable: OPTIMIZATION_CAPABILITIES : FP32 BIN FP16 INT8 EXPORT_IMPORT
[ INFO ]                 Immutable: GPU_DEVICE_TOTAL_MEM_SIZE : 62225842176
[ INFO ]                 Immutable: GPU_UARCH_VERSION : 785.192.4
[ INFO ]                 Immutable: GPU_EXECUTION_UNITS_COUNT : 112
[ INFO ]                 Immutable: GPU_MEMORY_STATISTICS : ""
[ INFO ]                 Mutable: PERF_COUNT : NO
[ INFO ]                 Mutable: MODEL_PRIORITY : MEDIUM
[ INFO ]                 Mutable: GPU_HOST_TASK_PRIORITY : MEDIUM
[ INFO ]                 Mutable: GPU_QUEUE_PRIORITY : MEDIUM
[ INFO ]                 Mutable: GPU_QUEUE_THROTTLE : MEDIUM
[ INFO ]                 Mutable: GPU_ENABLE_LOOP_UNROLLING : YES
[ INFO ]                 Mutable: GPU_DISABLE_WINOGRAD_CONVOLUTION : NO
[ INFO ]                 Mutable: CACHE_DIR : ""
[ INFO ]                 Mutable: CACHE_MODE : optimize_speed
[ INFO ]                 Mutable: PERFORMANCE_HINT : LATENCY
[ INFO ]                 Mutable: EXECUTION_MODE_HINT : PERFORMANCE
[ INFO ]                 Mutable: COMPILATION_NUM_THREADS : 18
[ INFO ]                 Mutable: NUM_STREAMS : 1
[ INFO ]                 Mutable: PERFORMANCE_HINT_NUM_REQUESTS : 0
[ INFO ]                 Mutable: INFERENCE_PRECISION_HINT : f16
[ INFO ]                 Mutable: ENABLE_CPU_PINNING : NO
[ INFO ]                 Mutable: DEVICE_ID : 0

[Aznie_Intel · Accepted solution]

Hi Matthew,

Thanks for reaching out. The supported platform for NPU devices is OS: Ubuntu* 22.04 64-bit (with Linux kernel 6.6+). The NPU Plugin needs an NPU Driver installed and NPU plugin (openvino_intel_npu_plugin.dll) is available.

NPU plugin is currently available only with the Archive distribution of OpenVINO™ (https://storage.openvinotoolkit.org/repositories/openvino/packages/2024.1/linux)

Also, the Intel® NPU device requires a proper driver to be installed in the system. You may refer to these Configurations for Intel® NPU with OpenVINO™. After that, you may check the NPU with the query device:

import openvino as ov

core = ov.Core()

core.available_devices

And, for device name:

device = "NPU"

core.get_property(device, "FULL_DEVICE_NAME")

Hope this helps.

Regards,

Aznie

[Matthew_C_Intel]

Thank you Aznie.

I have successfully enabled NPU access by re-reading the link you provided above and installing the missing NPU user-space drivers . Here were my missing steps:

wget https://github.com/intel/linux-npu-driver/releases/download/v1.2.0/intel-driver-compiler-npu_1.2.0.20240404-8553879914_ubuntu22.04_amd64.deb
wget https://github.com/intel/linux-npu-driver/releases/download/v1.2.0/intel-fw-npu_1.2.0.20240404-8553879914_ubuntu22.04_amd64.deb
wget https://github.com/intel/linux-npu-driver/releases/download/v1.2.0/intel-level-zero-npu_1.2.0.20240404-8553879914_ubuntu22.04_amd64.deb
sudo dpkg -i *.deb

There may be an opportunity to improve the OpenVINO NPU documentation at https://docs.openvino.ai/2024/get-started/configurations/configurations-intel-npu.html to clarify installation steps for both a user-space driver and kernel driver.   When I originally read this documentation, I followed the steps to verify the kernel driver and incorrectly assumed that the OpenVINO archive included everything else, thus neglecting to install the NPU user-space drivers.   

Specifically I would recommend changing "It is recommended to check for the latest version of the driver." to "In addition to the required user-space driver install above, it is also required to check for the latest version kernel driver using the steps below."

Hope this helps!

-Matt

[harpreetsahota]

Hi Matt,

I'm working on the same device as you and trying to accomplish the same thing. Can you confirm which distribution on Ubuntu and the Kernel version you used? I installed Ubuntu 22.04 on my NUC, which has Kernel version 6.5. I tried to upgrade to Kernel 6.6 via mainline but I keep running into a dependency issue with libc6...which isn't recommended to upgrade manually (and is proving difficult to do so even if I try)

[Matthew_C_Intel]

I used Ubuntu 24.04 and its default kernel with the dockerfile in my repo at https://github.com/mattcurf/npu_play for the user-space drivers and libraries.

-Matt

[Aznie_Intel]

Hi Matthew,

I am glad that your issue is resolved. I appreciate your suggestion for improving our documentation. Therefore, this thread will no longer be monitored since this issue has been resolved. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie