Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)
4974 Discussions

amplxe-gui crashes while Finalizing results / Resolving information for xxx

Ryan_B_
Beginner
599 Views
Kernel: Linux rbeasley-lx 3.2.0-57-generic #87-Ubuntu SMP Tue Nov 12 21:35:10 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Ubuntu 12.04.3 LTS Intel VTune Amplifier XE 2013 Update 15 I just began an evaluation of Intel VTune Amplifier XE 2013 Update 15 for Linux. After sampling my application by attaching to a running process, VTune is unable to ever perform an analysis of the samples. The application always ends up crashing during the "Finalizing results" phase. When crashing, the current library being resolved varies with each attempt. My machine has 18G of RAM with an add'l 8G of swap. (gdb) bt #0 0x00007fb4819c2015 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #1 0x00007fb4819bc187 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #2 0x00007fb4819bb6a5 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #3 0x00007fb4819df54a in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #4 0x00007fb4819df47c in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #5 0x00007fb4819def4a in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #6 0x00007fb4819a874f in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #7 0x00007fb483a16448 in dicerresolver_2_2::internal::ism_module_bank::get_symbol(dicerresolver_2_2::internal::segment_rva const&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #8 0x00007fb483a4743d in dicerresolver_2_2::internal::virtual_stack::virtual_stack(dicerresolver_2_2::internal::resolver_impl&, dbinterface_1_38::Index, dicerresolver_2_2::internal::segment_rva const&, boost::shared_ptr<:INTERNAL::MODULE_BANK>, dicerresolver_2_2::resolution_type_set&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #9 0x00007fb483a325d8 in dicerresolver_2_2::internal::resolver_impl::resolve_code_locations(dicerresolver_2_2::internal::attribute_row_ref_table*, dbinterface_1_38::Index, dbinterface_1_38::Index, boost::shared_ptr<:INTERNAL::MODULE_BANK>, msngr2::IProgress*, dicerresolver_2_2::resolution_type_set&, dicerresolver_2_2::internal::call_target_row_ref_table&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #10 0x00007fb483a42c02 in dicerresolver_2_2::internal::resolver_impl::resolve_module(dbinterface_1_38::Index, msngr2::IProgress*, dicerresolver_2_2::internal::call_target_row_ref_table&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #11 0x00007fb4837595ad in dicerresolver_2_2::internal::parallel_module_loading::do_resolve(msngr2::IProgress*, dbinterface_1_38::Index, dicerresolver_2_2::internal::call_target_row_ref_table&) const () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #12 0x00007fb483759e41 in dicerresolver_2_2::internal::parallel_module_loading::operator()(tbb::blocked_range<:INDEX> const&) const () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #13 0x00007fb483775344 in tbb::internal::start_for<:BLOCKED_RANGE><:INDEX>, dicerresolver_2_2::internal::parallel_module_loading, tbb::auto_partitioner>::execute() () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #14 0x00007fb492ba8911 in tbb::internal::custom_scheduler<:INTERNAL::INTELSCHEDULERTRAITS>::local_wait_for_all(tbb::task&, tbb::task*) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #15 0x00007fb492ba66fc in tbb::internal::arena::process(tbb::internal::generic_scheduler&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #16 0x00007fb492ba583c in tbb::internal::market::process(rml::job&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #17 0x00007fb492ba335b in tbb::internal::rml::private_worker::run() () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #18 0x00007fb492ba3266 in tbb::internal::rml::private_worker::thread_routine(void*) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #19 0x00007fb4a9d7ee9a in start_thread (arg=0x7fb479222700) at pthread_create.c:308 #20 0x00007fb4a84313fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #21 0x0000000000000000 in ?? () (gdb) info share /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so From To Syms Read Shared Object Library 0x00007fb483a0aa60 0x00007fb483a5a048 Yes (*) /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so (*): Shared library is missing debugging information. (gdb) frame 7 #7 0x00007fb483a16448 in dicerresolver_2_2::internal::ism_module_bank::get_symbol(dicerresolver_2_2::internal::segment_rva const&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so (gdb) info reg rax 0x1 1 rbx 0x3b170 242032 rcx 0x1f226 127526 rdx 0x1f21d 127517 rsi 0x7fb55deb53b0 140416941511600 rdi 0x7fb47921ec38 140413103107128 rbp 0x7fb470081ba0 0x7fb470081ba0 rsp 0x7fb47921f040 0x7fb47921f040 r8 0x1 1 r9 0x8 8 r10 0x7fb481c1c368 140413247800168 r11 0x1f21d 127517 r12 0x7fb47921fcb0 140413103111344 r13 0x7fb47921fc80 140413103111296 r14 0x7fb47007bb70 140412950395760 r15 0x7fb47007bb70 140412950395760 rip 0x7fb483a16448 0x7fb483a16448 <:INTERNAL::ISM_MODULE_BANK::GET_SYMBOL> eflags 0x202 [ IF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
0 Kudos
11 Replies
Ryan_B_
Beginner
599 Views
New analysis, similar crash. This time I launched VTune via gdb. beasley@rbeasley-lx ~> gdb amplxe-gui GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04 Copyright (C) 2012 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-linux-gnu". For bug reporting instructions, please see: <>... Reading symbols from /opt/intel/vtune_amplifier_xe_2013/bin64/amplxe-gui...(no debugging symbols found)...done. (gdb) r Starting program: /opt/intel/vtune_amplifier_xe_2013/bin64/amplxe-gui [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1". [New Thread 0x7fffe4cd3700 (LWP 29705)] [New Thread 0x7fffdffff700 (LWP 29706)] [New Thread 0x7fffdf1ca700 (LWP 29707)] [New Thread 0x7fffd77e1700 (LWP 29848)] [New Thread 0x7fffd6fe0700 (LWP 29849)] ** (amplxe-gui:29696): CRITICAL **: os_bar_hide: assertion `OS_IS_BAR (bar)' failed (amplxe-gui:29696): Gtk-CRITICAL **: IA__gtk_widget_hide: assertion `GTK_IS_WIDGET (widget)' failed ** (amplxe-gui:29696): CRITICAL **: os_bar_set_parent: assertion `OS_IS_BAR (bar)' failed ** (amplxe-gui:29696): CRITICAL **: os_bar_hide: assertion `OS_IS_BAR (bar)' failed (amplxe-gui:29696): Gtk-CRITICAL **: IA__gtk_widget_hide: assertion `GTK_IS_WIDGET (widget)' failed ** (amplxe-gui:29696): CRITICAL **: os_bar_set_parent: assertion `OS_IS_BAR (bar)' failed [New Thread 0x7fffc5522700 (LWP 30043)] [Thread 0x7fffdf1ca700 (LWP 29707) exited] [New Thread 0x7fffdf1ca700 (LWP 30100)] [New Thread 0x7fffbd880700 (LWP 30102)] [New Thread 0x7fffbdc81700 (LWP 30101)] [New Thread 0x7fffbcc7d700 (LWP 30105)] [New Thread 0x7fffbd07e700 (LWP 30104)] [New Thread 0x7fffbd47f700 (LWP 30103)] [New Thread 0x7fffbc87c700 (LWP 30106)] Program received signal SIGSEGV, Segmentation fault. [Switching to Thread 0x7fffbcc7d700 (LWP 30105)] 0x00007fffc6524015 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so (gdb) bt #0 0x00007fffc6524015 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #1 0x00007fffc651e187 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #2 0x00007fffc651d6a5 in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #3 0x00007fffc654154a in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #4 0x00007fffc654147c in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #5 0x00007fffc6540f4a in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #6 0x00007fffc650a74f in ?? () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so #7 0x00007fffc8578448 in dicerresolver_2_2::internal::ism_module_bank::get_symbol(dicerresolver_2_2::internal::segment_rva const&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #8 0x00007fffc85a943d in dicerresolver_2_2::internal::virtual_stack::virtual_stack(dicerresolver_2_2::internal::resolver_impl&, dbinterface_1_38::Index, dicerresolver_2_2::internal::segment_rva const&, boost::shared_ptr<:INTERNAL::MODULE_BANK>, dicerresolver_2_2::resolution_type_set&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #9 0x00007fffc85945d8 in dicerresolver_2_2::internal::resolver_impl::resolve_code_locations(dicerresolver_2_2::internal::attribute_row_ref_table*, dbinterface_1_38::Index, dbinterface_1_38::Index, boost::shared_ptr<:INTERNAL::MODULE_BANK>, msngr2::IProgress*, dicerresolver_2_2::resolution_type_set&, dicerresolver_2_2::internal::call_target_row_ref_table&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #10 0x00007fffc85a4c02 in dicerresolver_2_2::internal::resolver_impl::resolve_module(dbinterface_1_38::Index, msngr2::IProgress*, dicerresolver_2_2::internal::call_target_row_ref_table&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_impl_2.2.so #11 0x00007fffc82bb5ad in dicerresolver_2_2::internal::parallel_module_loading::do_resolve(msngr2::IProgress*, dbinterface_1_38::Index, dicerresolver_2_2::internal::call_target_row_ref_table&) const () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #12 0x00007fffc82bbe41 in dicerresolver_2_2::internal::parallel_module_loading::operator()(tbb::blocked_range<:INDEX> const&) const () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #13 0x00007fffc82d7344 in tbb::internal::start_for<:BLOCKED_RANGE><:INDEX>, dicerresolver_2_2::internal::parallel_module_loading, tbb::auto_partitioner>::execute() () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_dicerresolver_core_2.2.so #14 0x00007fffd7800911 in tbb::internal::custom_scheduler<:INTERNAL::INTELSCHEDULERTRAITS>::local_wait_for_all(tbb::task&, tbb::task*) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #15 0x00007fffd77fe6fc in tbb::internal::arena::process(tbb::internal::generic_scheduler&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #16 0x00007fffd77fd83c in tbb::internal::market::process(rml::job&) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #17 0x00007fffd77fb35b in tbb::internal::rml::private_worker::run() () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #18 0x00007fffd77fb266 in tbb::internal::rml::private_worker::thread_routine(void*) () from /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libtbb.so.2 #19 0x00007ffff434be9a in start_thread (arg=0x7fffbcc7d700) at pthread_create.c:308 #20 0x00007ffff29fe3fd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #21 0x0000000000000000 in ?? () (gdb) info reg rax 0x1 1 rbx 0x1a883 108675 rcx 0x1a883 108675 rdx 0x1a87a 108666 rsi 0x7fffa8bb7570 140736024245616 rdi 0x7fffbcc79c38 140736360586296 rbp 0x1a87a 0x1a87a rsp 0x7fffbcc79ae0 0x7fffbcc79ae0 r8 0x1 1 r9 0x8 8 r10 0x7fffc677e368 140736523133800 r11 0x1a87a 108666 r12 0x7fffbcc79c38 140736360586296 r13 0x8 8 r14 0x7fffa8bb7570 140736024245616 r15 0x1 1 rip 0x7fffc6524015 0x7fffc6524015 eflags 0x10202 [ IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0 (gdb) info share /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so From To Syms Read Shared Object Library 0x00007fffc64f3ef0 0x00007fffc6556278 Yes (*) /opt/intel/vtune_amplifier_xe_2013/lib64/../lib64/../lib64/libamplxe_ism_core_3.19.so (*): Shared library is missing debugging information.
0 Kudos
Peter_W_Intel
Employee
599 Views

Thanks for this valuable report.

Did you work on advanced-hotspots data collector? To simplify the problem, try command line:

amplxe-cl -collect advanced-hotspots -no-auto-finalize -duration 60 -target-pid PID

Then, you can try:

amplxe-cl -finalize -r r00?ah ; have you seen similar problem in GUI? What is error message?

0 Kudos
Peter_W_Intel
Employee
599 Views

I wonder that the tool will not crash when resolving symbol info:-)

0 Kudos
TimP
Honored Contributor III
599 Views

You must try to keep the amount of data collection within limits.  The sampling rate can be adjusted by command line sample-after value or by the estimated time in the Advanced section of the GUI setup.  The estimated time is self-explanatory for the case where there are perhaps up to 4 total threads running on the platform.  Beyond that, you must reduce estimated time to a fraction of real time.  The .tb6 file size limits were increased several updates back, but even 300MB may be excessive.

0 Kudos
Bernard
Valued Contributor I
599 Views

It seems that faulting IP is located at this code point rip 0x7fb483a16448 0x7fb483a16448 <dicerresolver_2_2::internal::ism_module_bank::get_symbol(dicerresolver_2_2::internal::segment_rva const&)+264>.

Now because this  function belongs to VTune library so the source code and signature will not be available.By looking at function description and if the error is signaled as a segfault I can theorize that probably the value passed by reference could point to non-accessible memory.It can be this value segment_rva const&.

As I do not have an experience with GDB so I would like to ask you to resolve with the help of GDB address which containes rsi register(rsi 0x7fb55deb53b0).

0 Kudos
Ryan_B_
Beginner
599 Views

Peter Wang (Intel) wrote:

Did you work on advanced-hotspots data collector? To simplify the problem, try command line:

amplxe-cl -collect advanced-hotspots -no-auto-finalize -duration 60 -target-pid PID

Then, you can try:

amplxe-cl -finalize -r r00?ah ; have you seen similar problem in GUI? What is error message?

Using the GUI, I've tried the Basic Hotspots, Advanced Hotspots, and General Exploration - Nehalem / Westmere analysis types. The Basic Hotspots analysis is the least flaky, but still fails for the same reason the majority of the time.

I followed your suggestions using the command line interface (amplxe-cl), and everything appears stable. The collect and finalize steps both succeeded. Next I changed into my the project folder for my application, and I reran "amplxe-cl -finalize" against each of them. The command consistently succeeded. :D

Going back to the original report about the GUI, the UI never displayed an explicit error message. It'd only trigger a SIGSEGV, which would be handled by the UI popping up a "would you like to submit this crash report to Intel?" dialog.

So, as of writing, I now have a workaround by dealing with sample collection via command line, and I'll open the results in the GUI for review. Especially considering the price attached to the package, I'd still prefer that the GUI not explode, however. ;)

Thanks for the assist!

0 Kudos
Ryan_B_
Beginner
599 Views

Tim Prince wrote:

You must try to keep the amount of data collection within limits.  The sampling rate can be adjusted by command line sample-after value or by the estimated time in the Advanced section of the GUI setup.  The estimated time is self-explanatory for the case where there are perhaps up to 4 total threads running on the platform.  Beyond that, you must reduce estimated time to a fraction of real time.  The .tb6 file size limits were increased several updates back, but even 300MB may be excessive.

That's a fair observation, however I should note that I'm sampling an application for less than a minute, and my sampled data (r?????/data.0) maxes out at around 25M for the longest sample duration. I don't think I was collecting excessive amounts of information. :) That said, I'd find it kinda ironic that a developer tool geared toward helping people write robust code crashes 'cuz it may have been fed too much data. (And it's not like some 3rd party program generated the input data!)

iliyapolak wrote:

As I do not have an experience with GDB so I would like to ask you to resolve with the help of GDB address which containes rsi register(rsi 0x7fb55deb53b0).

gdb was unable to resolve the address to anything. :\

0 Kudos
Bernard
Valued Contributor I
599 Views

>>>gdb was unable to resolve the address to anything. :\>>>

Maybe this is the culprit?Btw can explain what is the error code?Is that segfault?

0 Kudos
Bernard
Valued Contributor I
599 Views

>>>Btw can explain what is the error code?Is that segfault?>>>

Please disregard this quoted sentence,because I have found the error description in your first post.

Regarding the mentioned  address which holds rsi register I meant possibility to resolve the address to some memory region like process heap.

0 Kudos
Vitaly_S_Intel
Employee
599 Views

Hello Ryan!

When GUI handled SIGSEGV, were you able to submit crash report? That could help us to find the reason of GUI problem.

0 Kudos
Ryan_B_
Beginner
599 Views

Vitaly Slobodskoy (Intel) wrote:

Hello Ryan!

When GUI handled SIGSEGV, were you able to submit crash report? That could help us to find the reason of GUI problem.

I avoided sending the crash reports, because I was concerned that doing so might leak confidential information. I'm working on another task now, but upon switching back I'll give this a shot. Thanks!

0 Kudos
Reply