topic Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots in Analyzers

[VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Sat, 29 Apr 2023 19:50:56 GMT

When running vtune-self-checker.sh:

GPU software event-based analysis with runtime tracing Example of analysis types: GPU Compute/Media Hotspots (source analysis mode) Collection: Fail vtune: Warning: To enable hardware event-based sampling, VTune Profiler has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes. The check observed a product failure on your system. Review errors in the output above to fix a problem or contact Intel technical support. The system is ready for the following analyses: * Performance Snapshot * Hotspots and Threading with user-mode sampling * Hotspots with HW event-based sampling, HPC Performance Characterization, etc. * Microarchitecture Exploration * Memory Access * Hotspots with HW event-based sampling and call stacks * Threading with HW event-based sampling * GPU HW event-based analysis with runtime tracing The following analyses have failed on the system: * GPU Compute/Media Hotspots (source analysis mode) Log location: /tmp/vtune-tmp-bram/self-checker-2023.04.29_11.18.24/log.txt

Full log is attached.

It just says: "Collection: Fail" with no additional error messages, other than an NMI warning.

This is on:

OS: Ubuntu 23.04

vtune: 2023.1.0 pre-release (build 625246)

CPU: i12600k

GPU: Intel AlderLake-S GT1

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Tue, 02 May 2023 15:37:53 GMT

Hi,

Thank you for posting in Intel communities.

We assume the matrix.dpcpp(binary, which is used in the self_check.py) sample is unable to recognize the GPU device.

Please follow the below steps.

We assume you installed Basetoolkit. Please set environment variables with the below command.

source /opt/intel/oneapi/setvars.sh

To check available devices, please run the below command

sycl-ls

The output looks like the below screenshot. (you can see GPU details if GPU is available)

As you raised three threads about this issue, can we go ahead and close this thread so we can discuss it there (https://community.intel.com/t5/Analyzers/VTUNE-self-checker-needs-updating-from-SYCL-to-ONEAPI/m-p/1481576#M23364)?

If this resolves your issue, make sure to accept this as a solution. This would help others with similar issues. Thank you!

Regards,

Jaideep

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Tue, 02 May 2023 16:01:19 GMT

I have no sycl-ls, even though intel-basekit-getting-started is installed, as is intel-basekit-runtime-2023.1.0

bram@deca:~$ source /opt/intel/oneapi/setvars.sh :: initializing oneAPI environment ... bash: BASH_VERSION = 5.2.15(1)-release args: Using "$@" for setvars.sh arguments: :: compiler -- latest :: dev-utilities -- latest :: dnnl -- latest :: tbb -- latest :: vtune -- latest :: oneAPI environment initialized :: bram@deca:~$ dpkg --list | grep intel- ii intel-basekit-getting-started 2023.1.0-46401 all Intel® oneAPI Base Toolkit ii intel-basekit-runtime-2023.1.0 2023.1.0-46401 amd64 Intel® oneAPI Base Toolkit ii intel-gpu-tools 1.27.1-1 amd64 tools for debugging the Intel graphics driver ii intel-media-va-driver:amd64 23.1.2+dfsg1-1 amd64 VAAPI driver for the Intel GEN8+ Graphics family ii intel-microcode 3.20230214.0ubuntu1 amd64 Processor microcode firmware for Intel CPUs ii intel-oneapi-common-licensing-2023.1.0 2023.1.0-43473 all oneAPI Common License ii intel-oneapi-common-vars 2023.1.0-43473 all oneAPI Common Environment Scripts ii intel-oneapi-compiler-cpp-eclipse-cfg 2023.1.0-46305 all Intel® oneAPI DPC++/C++ Compiler 2023.1.0 for Linux* eclipse integration configuration file (C++) ii intel-oneapi-compiler-dpcpp-cpp-common-2023.1.0 2023.1.0-46305 all Intel® oneAPI DPC++/C++ Compiler 2023.1.0 for Linux* ii intel-oneapi-compiler-dpcpp-cpp-runtime 2023.1.0-46305 amd64 Intel® oneAPI DPC++/C++ Compiler 2023.1.0 for Linux* runtime package for Intel(R) 64 ii intel-oneapi-compiler-dpcpp-cpp-runtime-2023.1.0 2023.1.0-46305 amd64 Intel® oneAPI DPC++/C++ Compiler 2023.1.0 for Linux* runtime package for Intel(R) 64 ii intel-oneapi-compiler-dpcpp-eclipse-cfg 2023.1.0-46305 all Intel® oneAPI DPC++/C++ Compiler 2023.1.0 for Linux* eclipse integration configuration file (DPC++) ii intel-oneapi-compiler-shared-runtime-2023.1.0 2023.1.0-46305 amd64 Intel(R) Compiler Shared Files runtime contents ii intel-oneapi-condaindex 2023.1.0-43291 amd64 oneAPI Common Install components ii intel-oneapi-dev-utilities-2021.9.0 2021.9.0-44447 amd64 Dev Utilities ii intel-oneapi-dev-utilities-eclipse-cfg 2021.9.0-44447 all intel-oneapi-dev-utilities-eclipse-cfg ii intel-oneapi-dnnl 2023.1.0-46343 amd64 Intel® oneAPI Deep Neural Network Library ii intel-oneapi-icc-eclipse-plugin-cpp-2023.1.0 2023.1.0-46305 all Standards driven high performance cross architecture compiler and high performance C++ CPU focused compiler ii intel-oneapi-openmp-2023.1.0 2023.1.0-46305 amd64 Intel® OpenMP* Runtime Library 2023.1.0 for Linux* for Intel(R) 64 ii intel-oneapi-openmp-common-2023.1.0 2023.1.0-46305 all Intel® OpenMP* Runtime Library 2023.1.0 for Linux* ii intel-oneapi-runtime-opencl 2023.1.0-46305 amd64 Intel® CPU Runtime for OpenCL(TM) Applications runtime ii intel-oneapi-runtime-tbb 2021.9.0-43484 amd64 Intel® oneAPI Threading Building Blocks runtime ii intel-oneapi-runtime-tbb-common 2021.9.0-43484 all Intel® oneAPI Threading Building Blocks runtime common ii intel-oneapi-tbb-2021.9.0 2021.9.0-43484 amd64 Intel® oneAPI Threading Building Blocks ii intel-oneapi-tbb-common-2021.9.0 2021.9.0-43484 all Intel® oneAPI Threading Building Blocks common ii intel-oneapi-vtune 2023.1.0-44286 amd64 Intel® VTune(TM) Profiler ii intel-oneapi-vtune-eclipse-plugin-vtune 2023.1.0-44286 all Locate and optimize performance bottlenecks fast across CPU, GPU, and FPGA systems. ii intel-opencl-icd 22.43.24595.41-1 amd64 Intel graphics compute runtime for OpenCL ii libdrm-intel1:amd64 2.4.115+git2302101558.332809f3~j~mesarc0 amd64 Userspace interface to intel-specific kernel DRM services -- runtime bram@deca:~$ sycl-ls sycl-ls: command not found

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Tue, 02 May 2023 16:01:41 GMT

Hi,

Can you install Basetoolkit with the below command and let us know the output of sycl-ls.

wget https://registrationcenter-download.intel.com/akdlm/IRC_NAS/7deeaac4-f605-4bcf-a81b-ea7531577c61/l_BaseKit_p_2023.1.0.46401_offline.sh

sycl-ls

Thanks,

Jaideep

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Tue, 02 May 2023 16:09:51 GMT

Thank you JaideepK,

Can you please tell me in which .deb package sycl-ls is located?

I am usin the intel package repository, and I prefer to install via packages, not via an .sh script.

$ cat /etc/apt/sources.list.d/oneAPI.list deb [signed-by=/usr/share/keyrings/oneapi-archive-keyring.gpg] https://apt.repos.intel.com/oneapi all main

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Tue, 02 May 2023 16:15:46 GMT

Sorry for the duplicate post, the forum somehow lost my first one after adding a label, so I redid it.

The solution to this selfcheck fail is to preload libpthread, as described here.

Re:[VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Thu, 04 May 2023 06:11:29 GMT

Hi,

I hope you are doing well.

Sorry to say this, but the OS that you are using is not supported for Vtune. Could you please try on a supported operating system?

If you want to try Ubuntu 22, please follow the documentation below.

https://dgpu-docs.intel.com/installation-guides/ubuntu/ubuntu-jammy-arc.html

You can directly profile any GPU binary with the below command:

example: vtune -collect gpu-hotspots <Path to the GPU Binary>

Thanks,

Jaideep

Re: Re:[VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Thu, 04 May 2023 22:07:24 GMT

Thank you Jaideep,

So, downgrading the OS is quite involved, so I will skip that. However, using

/opt/intel/oneapi/diagnostics/diagnostics.py --force --filter vtune_sys_check -vv

I was able to learn that:

CONFIG_DRM_I915_LOW_LEVEL_TRACEPOINTS---------------------Disable or not set

I am currently following your guide on kernel recompilation and will try doing that. If that fails too, I will give up.

UPDATE: After building kernel with correct configuration, the diagnostics error went away, but still no gpu-hotspot samples recorded.

was disabled because it doesn't support the given collection type.

GPU device is seen, though:

# sycl-ls [opencl:acc:0] Intel(R) FPGA Emulation Platform for OpenCL(TM), Intel(R) FPGA Emulation Device 1.2 [2023.15.3.0.20_160000] [opencl:cpu:1] Intel(R) OpenCL, 12th Gen Intel(R) Core(TM) i5-12600K 3.0 [2023.15.3.0.20_160000] [opencl:gpu:2] Intel(R) OpenCL HD Graphics, Intel(R) UHD Graphics 770 [0x4680] 3.0 [22.43.24595] [ext_oneapi_level_zero:gpu:0] Intel(R) Level-Zero, Intel(R) UHD Graphics 770 [0x4680] 1.3 [1.3.24595]

Bram

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Fri, 05 May 2023 04:44:51 GMT

Hi,

It's good to see that sycl-ls is showing GPU devices. Now you can run Vtune with any OneAPI GPU samples.

Please ignore the self-checker log. I think your machine is ready to go.

If you need any API samples, you can download them with the below command (you need to source environment variables).

oneapi-cli

If your issue is resolved, Can we go ahead and close this case?

Thanks,

Jaideep

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Fri, 05 May 2023 15:29:17 GMT

Nope... the matrix sample is still not working.

Finalization completed with warnings May 05 2023 08:22:28 Result finalization has completed with warnings that may affect the representation of the analysis data. Please see details below. Cannot load data file `/home/bram/intel/vtune/projects/sample (matrix)/r002gh/data.0/5449-5455.0.trace' (Data file is corrupted). Cannot locate debugging information for file `/lib/modules/6.2.6/kernel/drivers/nvme/host/nvme.ko'. Cannot read load addresses of sections from `/sys/module/nvme/sections'. This may affect the correctness of symbol resolution for `nvme'. Make sure this directory exists and all files in this directory have read permissions. Cannot locate debugging information for file `/home/intel/oneapi/vtune/2023.1.0/lib64/libtpsstool.so'. Cannot locate debugging information for the Linux kernel. Source-level analysis will not be possible. Function-level analysis will be limited to kernel symbol tables. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Tue, 09 May 2023 17:04:23 GMT

Hi,

I hope you are doing well.

We were able to run the Vtune self-checker without any issues after following the below workaround.

Could you please run the below commands as sudo/root user?

Setting the environment variables.

source /opt/intel/oneapi/setvars.sh

Installing dependencies (libdrm)

sudo apt install libdrm-dev sudo apt-get update

Cloning the repo and building packages.

git clone https://github.com/intel/metrics-discovery.git cd metrics-discovery mkdir build && cd build cmake ..

make -j$(nproc) sudo make install make package

If this resolves your issue, make sure to accept this as a solution. This would help others with similar issues.

Have a great day ahead

Thanks,

Jaideep

Re: [VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

Bram_S_ — Tue, 09 May 2023 17:23:03 GMT

You can close this,

i had already installed that metrics from git, which did not solve it.

I am on unsupported 23.04 so you can close this issue.

Re:[VTUNE] self-check collection-fail on GPU Computer/Media Hotspots

JaideepK_Intel — Wed, 10 May 2023 16:52:30 GMT

Hi,

If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.

Thanks,

Jaideep