topic ResNet50 Workload - Cannot locate debugging symbols in Analyzers

ResNet50 Workload - Cannot locate debugging symbols

Anonymous102 — Wed, 26 Feb 2020 18:57:16 GMT

Hi,

I am trying to get the VTune hotspots for a ResNet-50 workload written in Python. I am running the latest VTune (2020 version) on Ubuntu 18.04.

A few of the warnings seen are as follows:

vtune: Warning: Cannot locate file `<frozen importlib._bootstrap>'.
vtune: Warning: Cannot locate file `<frozen importlib._bootstrap_external>'.
vtune: Warning: Cannot locate file `<string>.ko'.
vtune: Warning: Cannot locate file `/tmp/tmpy52up4de.py'.
vtune: Warning: Cannot locate file `/tmp/tmpbbbewbwv.py'.
vtune: Warning: Cannot locate debugging information for file `/mnt/nvme_nfs_share/AIBench_shared_disk/Extra/vtune_profiler_2020.0.0.605129/bin64/pinbin'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/h5py/h5.cpython-36m-x86_64-linux-gnu.so'.
vtune: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libpthread.so.0'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/google/protobuf/pyext/_message.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/conversion.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libdl.so.2'.
Cannot match the module with the symbol file `/lib/x86_64-linux-gnu/libdl-2.27.so'. Make sure to specify the correct path to the symbol file in the Binary/Symbol Search list of directories.
vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/timezones.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/c_timestamp.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/mnt/nvme_nfs_share/AIBench_shared_disk/Extra/vtune_profiler_2020.0.0.605129/lib64/libtpsstool.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/_solib_k8/_U@mkl_Ulinux_S_S_Cmkl_Ulibs_Ulinux___Uexternal_Smkl_Ulinux_Slib/libiomp5.so'.

Please find a screenshot of the hotspots on the VTune GUI attached.

As seen form the screenshot, the most time consuming hotspot is not detected.

Please can I get some help fixing this.

Hi,

Anonymous102 — Thu, 27 Feb 2020 00:02:37 GMT

Hi,

Just an update: I installed the debug version of Python 3.6 and then ran VTune. I still see these warnings.

Please help.

If kernel pointers

ArunJ_Intel — Thu, 27 Feb 2020 13:18:16 GMT

If kernel pointers information was explicitly hidden by setting the kptr_restrict to a non-zero value, hardware event-based analysis results may not contain functions from kernel modules. As a result, you may see the CPU time associated with the [Outside any known module] item. To workaround this problem for the current session, set the contents of the /proc/sys/kernel/kptr_restrict file to 0 before starting the VTune Profiler as follows:

sysctl -w kernel.kptr_restrict=0

I have already set the

Anonymous102 — Fri, 28 Feb 2020 17:51:14 GMT

I have already set the contents of the /proc/sys/kernel/kptr_restrict file to 0. Still I am seeing CPU time associated with [Outside any known module].

Is there anything else to fix?

Just adding more details:

Anonymous102 — Fri, 28 Feb 2020 18:07:59 GMT

Just adding more details:

Ubuntu 18.04

Kernel Version: 5.4.13

I am unable to find the kernel compressed image with debug symbols for this kernel version.

We are forwarding your case

ArunJ_Intel — Tue, 03 Mar 2020 09:30:43 GMT

We are forwarding your case for SME to check

Arun Jose

Hi Neha,

Mayank_T_Intel — Wed, 08 Apr 2020 11:32:21 GMT

Hi Neha,

Can you confirm the collector type? If it is the perf collector, then you need to set /sys/kernel/perf_event_paranoid to <=1 to profile kernel.

Thanks,

Mayank

Re:ResNet50 Workload - Cannot locate debugging sym...

DrAmarpal_K_Intel — Wed, 05 May 2021 06:45:35 GMT

Hi Neha,

Having not received your response we are assuming that the issue has been fixed and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.