Vtune Profiler failing to profile NPU programs

[Martin_HZK]

I am trying to perform the NPU Exploration analysis. The NPU test program is the benchmark_app in OpenVINO[Sample] . I then use Vtune Profiler to do the NPU Exploration analysis. I set the Application parameters:

-m {model_path} -d NPU -hint latency

And the wrapper script is like this. I activate the required conda environment and the source the variables.

rem Prefix: Set up environment
call C:\Users\xiaor\miniconda3\condabin\conda.bat activate vino
call "C:\Program Files (x86)\Intel\openvino_2024\setupvars.bat"


echo "Environment initialized."

rem Run VTune collector
%*

rem Postfix: Process results
dir %VTUNE_RESULT_DIR%

However, it turns out that the analysis failed to run and the NPU is not running any tasks. But this same program is tested to run successfully on NPU in cmd. 

Additionally, I tested NPU with Performance Snapshot, it turns out that the inference is running successfully with the desired output, indicating that my wrapper script and application parameters are working successfully. So I reckon it is the problem with the NPU Exploration section. Any possible adjustments recommended?

And after running the NPU Exploration, the available profiling results are Hotspots by CPU Utilization and Threading Efficiency. Why is that?

OS: Windows *11 23H2

OpenVINO：2024.5.0 (Installed from archive)

Vtune: Installed with oneAPI 2025.0

[yuzhang3_intel]

Could you share the VTune data?

[Martin_HZK]

What do you mean by the Vtune data? The raw testing exe file or the wrapper script?

[yuzhang3_intel]

Even if the profiling failed, there should be a VTune data folder generated in the current working directory, like r000ne, r000ps, etc.

[Martin_HZK]

This is one of the NPU Exploration analyses.

[yuzhang3_intel]

There are some error message from the log in r019npu data. If you run the benchmark and options below without VTune, it can work well?

C:\Users\xiaor\miniconda3\envs\vino\Scripts\benchmark_app.exe -m C:\Program Files (x86)\Intel\openvino_2024\samples\cpp\model\ir_model\resnet50_fp16.xml -d NPU -hint latency

[Martin_HZK]

If I run the given shell command as you mentioned in the command line, then it shall be fine. Additionally, the same Launch Application setting in vtune is set to work fine for NPU in other profiling tasks including Performance Snapshot. The same Performance Snapshot profiling on CPU and GPU are also successfully performed only by switching 'NPU' ,the device name, correspondingly .

[yuzhang3_intel]

Need to do deeper investigation, could you share the model file?

[Martin_HZK]

Sure, I shall send you the model in an hour. Btw, do you have any sample for NPU Exploration analysis that is tested to work properly?

[Martin_HZK]

This is the XML model file.

[yuzhang3_intel]

I  think benchmark app is enough for verifying NPU profiling.

Could you share bin file of the model as well?

[Martin_HZK]

The bin file exceeded the limited size and here is the test.py file that utilize OpenVINO that generates both model file

[Arun11]

Hi Martin,

If you run the VTune benchmark app against the OpenVINO workload resnet50 would have resolved your issues. If I understand correctly that you can able to run NPU in command prompt but not in VTune GUI?.

[Martin_HZK]

> If I understand correctly that you can able to run NPU in command prompt but not in VTune GUI?.

For this question, yes.

But I didn't really understand the first sentence tbh

[Arun11]

I meant this command below, which is already suggested by yuzhang3_intel community.intel.com/t5/Analyzers/Vtune-Profiler-failing-to-profile-NPU-programs/m-p/1647303#M25452

[Martin_HZK]

Yes, this shell script works fine without Vtune.

[Arun11]

I hope the issue which you are facing is resolved.

[Martin_HZK]

No really My target is to utilize the Vtune to profile this benchmark_app, but the point is that I cannot run inside the Vtune profiler GUI. The command works fine for cli only without any vtune profiling command involved.

[Arun11]

1. Run the NPU workload. Ex: openVINO
2. Please verify whether the NPU is utilized in your system (Taskmanager-> performance) . If NPU utilization is zero than you have to run the workload properly.
3. Run the NPU exploration in VTune with the below settings
   1. goto launch application
   2. application : benchmark_app
   3. application parameters : -m "full path to the resnet50_fp16.xml" -d NPU
   4. select collection modes as Query-based in NPU exploration.
   5. run it
4. Once its done you can see the summary with the NPU device load and Compute Task

[Martin_HZK]

Thank you for your reply. Currently, I resolved the issue by downgrading the OpenVINO version from 2025.0 to 2024.4 as required by the NPU driver.

Hence, I strongly recommended that the OpenVINO toolkit add the related NPU driver dependencies into the documentation.

[yuzhang3_intel]

So the root cause is the NPU driver doesn't work well, right? As you mentioned benchmark_app works fine without VTune profiling.