Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)
5136 Discussions

ResNet50 Workload - Cannot locate debugging symbols

Anonymous102
Employee
2,419 Views

Hi,

I am trying to get the VTune hotspots for a ResNet-50 workload written in Python. I am running the latest VTune (2020 version) on Ubuntu 18.04.

 

A few of the warnings seen are as follows:

vtune: Warning: Cannot locate file `<frozen importlib._bootstrap>'.
vtune: Warning: Cannot locate file `<frozen importlib._bootstrap_external>'.
vtune: Warning: Cannot locate file `<string>.ko'.
vtune: Warning: Cannot locate file `/tmp/tmpy52up4de.py'.
vtune: Warning: Cannot locate file `/tmp/tmpbbbewbwv.py'.
vtune: Warning: Cannot locate debugging information for file `/mnt/nvme_nfs_share/AIBench_shared_disk/Extra/vtune_profiler_2020.0.0.605129/bin64/pinbin'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/h5py/h5.cpython-36m-x86_64-linux-gnu.so'.
vtune: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libpthread.so.0'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/google/protobuf/pyext/_message.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/conversion.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libdl.so.2'.
Cannot match the module with the symbol file `/lib/x86_64-linux-gnu/libdl-2.27.so'. Make sure to specify the correct path to the symbol file in the Binary/Symbol Search list of directories.
vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/timezones.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/pandas/_libs/tslibs/c_timestamp.cpython-36m-x86_64-linux-gnu.so'.

vtune: Warning: Cannot locate debugging information for file `/mnt/nvme_nfs_share/AIBench_shared_disk/Extra/vtune_profiler_2020.0.0.605129/lib64/libtpsstool.so'.

vtune: Warning: Cannot locate debugging information for file `/usr/local/lib/python3.6/dist-packages/_solib_k8/_U@mkl_Ulinux_S_S_Cmkl_Ulibs_Ulinux___Uexternal_Smkl_Ulinux_Slib/libiomp5.so'.
 

Please find a screenshot of the hotspots on the VTune GUI attached.

As seen form the screenshot, the most time consuming hotspot is not detected.

Please can I get some help fixing this.

 

0 Kudos
7 Replies
Anonymous102
Employee
2,419 Views

Hi,

Just an update: I installed the debug version of Python 3.6 and then ran VTune. I still see these warnings.

Please help.

0 Kudos
ArunJ_Intel
Moderator
2,419 Views

If kernel pointers information was explicitly hidden by setting the kptr_restrict to a non-zero value, hardware event-based analysis results may not contain functions from kernel modules. As a result, you may see the CPU time associated with the [Outside any known module] item. To workaround this problem for the current session, set the contents of the /proc/sys/kernel/kptr_restrict file to 0 before starting the VTune Profiler as follows:

sysctl -w kernel.kptr_restrict=0

 

0 Kudos
Anonymous102
Employee
2,419 Views

I have already set the contents of the /proc/sys/kernel/kptr_restrict file to 0. Still I am seeing CPU time associated with [Outside any known module].

Is there anything else to fix?

0 Kudos
Anonymous102
Employee
2,419 Views

Just adding more details:

Ubuntu 18.04

Kernel Version: 5.4.13

I am unable to find the kernel compressed image with debug symbols for this kernel version.

 

0 Kudos
ArunJ_Intel
Moderator
2,419 Views

We are forwarding your case for SME to check

 

Arun Jose

0 Kudos
Mayank_T_Intel
Employee
2,419 Views

Hi Neha,

Can you confirm the collector type? If it is the perf collector, then you need to set /sys/kernel/perf_event_paranoid to <=1 to profile kernel.

Thanks,

Mayank

0 Kudos
DrAmarpal_K_Intel
2,158 Views

Hi Neha,


Having not received your response we are assuming that the issue has been fixed and we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.



0 Kudos
Reply