Collection Failed when running XE2017 for C++ hotspots analysis

Jie_L_ · ‎06-13-2017

Failed to run the target in both GUI and command line.

Errors I got:

amplxe: Warning: Cannot stop posix timer: __NR_timer_settime() system call returned -1.
amplxe: Warning: Cannot stop posix timer: __NR_timer_settime() system call returned -1.
amplxe: Error: [Instrumentation Engine]: Source/pin/elfio/img_elf.cpp: ProcessSectionHeaders: 601: assertion failed: SEC_vaddr_i(sec) >= IMG_seg_text_vaddr_i(img) &SEC_vaddr_i(sec) < IMG_seg_data_vaddr_i(img)
amplxe: Collection failed.
amplxe: Internal Error

My program is a C++ QT Application running on Linux.

Can anyone help me. Thank you.

James_S · ‎06-13-2017

Hi Jie,

What VTune version are you using? Can you please share the crash report with the following command and attach the report.zip ?

amplxe-feedback -create-bug-report report.zip

Thanks,

Zhuowei

Jie_L_ · ‎06-14-2017

Hi Zhuowei,

The version is 2017 update 3.

Report is attached. Thanks

James_S · ‎06-20-2017

Hi Jie,

Can you please provide your Linux version, the command to run your application, and the full fail log? Thank you very much.

Best regards

Zhuowei

Milian_W_ · ‎06-26-2017

I'm running into the same issue with these Linux versions:

ArchLinux: 4.10.13

Ubuntu 17.04: 4.8.0 and 4.10.0

OpenSuse 42.1: 4.7.4 or 4.8.0 (unsure which one was used back then)

I don't think it's a Linux version thing, rather I suspect an update in GCC or Clang that introduces a change in the ELF header format? Maybe related to some kind of page protection/randomization feature?

Note also that this affects any tracing-related VTune analysis. The General Exploration or Advanced Hotspot analyses work fine for me.

Milian_W_ · ‎06-26-2017

Thinking about it, I actually think it's an issue with whatever compiler flag differences got applied to the Qt libraries. Afaik 5.6 works fine with VTune, but 5.8 breaks. I'll try to bisect it and see if I can figure out the exact reason for this failure on VTune's side. One way or another, there's a bug in VTune.

Milian_W_ · ‎06-26-2017

OK, so this is super strange... Running a simple Qt application against the system Qt 5.7.1 on a Ubuntu 17.04 with the 4.9'er kernel works. But using any of the official Qt binaries breaks VTune...

#include <QCoreApplication>
#include <iostream>

int main(int argc, char** argv)
{
  QCoreApplication app(argc, argv);
  std::cerr << "hello world" << std::endl;
  return 0;
}

Compile and run VTune against system Qt 5.7:

trainer@Trainer-PC:/tmp$ g++ test.cpp -std=c++11 -I/usr/include/x86_64-linux-gnu/qt5 -I/usr/include/x86_64-linux-gnu/qt5/QtCore -lQt5Core -fPIC -o a.systemqt
trainer@Trainer-PC:/tmp$ ldd a.systemqt 
        linux-vdso.so.1 =>  (0x00007fff597f2000)
        libQt5Core.so.5 => /usr/lib/x86_64-linux-gnu/libQt5Core.so.5 (0x00007f84ff062000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f84fecda000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f84feac3000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f84fe6fc000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f84fe4de000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f84fe2c0000)
        libicui18n.so.57 => /usr/lib/x86_64-linux-gnu/libicui18n.so.57 (0x00007f84fde46000)
        libicuuc.so.57 => /usr/lib/x86_64-linux-gnu/libicuuc.so.57 (0x00007f84fda9e000)
        libpcre16.so.3 => /usr/lib/x86_64-linux-gnu/libpcre16.so.3 (0x00007f84fd835000)
        libdouble-conversion.so.1 => /usr/lib/x86_64-linux-gnu/libdouble-conversion.so.1 (0x00007f84fd624000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f84fd420000)
        libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f84fd10a000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f84fcf02000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f84fcbf9000)
        /lib64/ld-linux-x86-64.so.2 (0x000056375db4f000)
        libicudata.so.57 => /usr/lib/x86_64-linux-gnu/libicudata.so.57 (0x00007f84fb17c000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f84faf09000)
trainer@Trainer-PC:/tmp$ /opt/intel/vtune_amplifier_xe_2017.3.0.510739/bin64/amplxe-cl -collect hotspots -app-working-dir $PWD -- ./a.out 
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /tmp/r006hs -command stop.
hello world
amplxe: Collection stopped.
amplxe: Using result path `/tmp/r006hs'
amplxe: Executing actions 75 % Generating a report                             
Collection and Platform Info
----------------------------
Parameter                 r006hs                                                                                                                                                                                                                                                                                                                                                                                            
------------------------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Application Command Line  ./a.out                                                                                                                                                                                                                                                                                                                                                                                           
Operating System          4.8.0-51-generic NAME="Ubuntu"
VERSION="17.04 (Zesty Zapus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 17.04"
VERSION_ID="17.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=zesty
UBUNTU_CODENAME=zesty
Computer Name             Trainer-PC                                                                                                                                                                                                                                                                                                                                                                                        
Result Size               2372764                                                                                                                                                                                                                                                                                                                                                                                           
Collection start time     14:16:12 26/06/2017 UTC                                                                                                                                                                                                                                                                                                                                                                           
Collection stop time      14:16:13 26/06/2017 UTC                                                                                                                                                                                                                                                                                                                                                                           

CPU
---
Parameter          r006hs                                        
-----------------  ----------------------------------------------
Name               Intel(R) Core(TM) Processor code named Haswell
Frequency          3399990480                                    
Logical CPU Count  4                                             

Summary
-------
Elapsed Time:  0.001
CPU Time:           
amplxe: Executing actions 100 % done

Now try the same but this time against an official Qt as downloaded from https://download.qt.io/archive/qt/5.6/5.6.2/qt-opensource-linux-x64-5.6.2.run.mirrorlist

trainer@Trainer-PC:/tmp$ g++ test.cpp -std=c++11 -I/home/trainer/Qt/qt-5.6.2/5.6/gcc_64/include -I/home/trainer/Qt/qt-5.6.2/5.6/gcc_64/include/QtCore -L/home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib -lQt5Core -fPIC
trainer@Trainer-PC:/tmp$ export LD_LIBRARY_PATH=/home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib
trainer@Trainer-PC:/tmp$ ./a.out 
hello world
trainer@Trainer-PC:/tmp$ /opt/intel/vtune_amplifier_xe_2017.3.0.510739/bin64/amplxe-cl -collect hotspots -app-working-dir $PWD -- ./a.out 
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /tmp/r005hs -command stop.
amplxe: Error: [Instrumentation Engine]: Source/pin/elfio/img_elf.cpp: ProcessSectionHeaders: 601: assertion failed: SEC_vaddr_i(sec) &gt;= IMG_seg_text_vaddr_i(img) &amp;&amp; SEC_vaddr_i(sec) &lt; IMG_seg_data_vaddr_i(img) 
amplxe: Collection failed.
amplxe: Internal Error
$ ldd a.out 
        linux-vdso.so.1 =>  (0x00007ffccf90d000)
        libQt5Core.so.5 => /home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib/libQt5Core.so.5 (0x00007f6e3ad0b000)
        libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f6e3a950000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f6e3a739000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f6e3a372000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f6e3a154000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f6e39f36000)
        libicui18n.so.56 => /home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib/libicui18n.so.56 (0x00007f6e39a9c000)
        libicuuc.so.56 => /home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib/libicuuc.so.56 (0x00007f6e396e4000)
        libicudata.so.56 => /home/trainer/Qt/qt-5.6.2/5.6/gcc_64/lib/libicudata.so.56 (0x00007f6e37d01000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f6e37afd000)
        libgthread-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libgthread-2.0.so.0 (0x00007f6e378fb000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f6e376f1000)
        libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007f6e373dd000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f6e370d4000)
        /lib64/ld-linux-x86-64.so.2 (0x000055db2ac72000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007f6e36e61000)

If it helps, I can attach the binaries. In general, the above steps should be enough information for someone to reproduce the issue though.

Vladimir_R_Intel · ‎06-27-2017

Hi Jie, Milian,

This issue should be fixed in the upcoming Vtune AXE 2017 U4 release.

BR,

Vladimir

Vladimir_R_Intel · ‎06-27-2017

Jie, Milian,

Could you please file Online Service Center (https://supporttickets.intel.com/?lang=en-US) issue with request for trying updated version of VTune with a fix (you need to refer on this forum thread in OSC request)?

In this case we will be able to provide you a pre-release VTune package to be sure that this issue is fixed.

BR,

Vladimir

Milian_W_ · ‎06-27-2017

Thanks, I'll do that now. Any ETA on the official release for U4?

Vladimir_R_Intel · ‎06-27-2017

It should be released in July.

Milian_W_ · ‎06-28-2017

The problem persists and is not fixed.

Collection failed
    Wed 28 Jun 2017 09:53:57 AM CEST  Collection failed. The data cannot be displayed. 
    [Instrumentation Engine]: Source/pin/elfio/img_elf.cpp: ProcessSectionHeaders: 601: assertion failed: SEC_vaddr_i(sec) &gt;= IMG_seg_text_vaddr_i(img) &amp;&amp; SEC_vaddr_i(sec) &lt; IMG_seg_data_vaddr_i(img)

Vladimir_R_Intel · ‎06-28-2017

Hi Milian,

you need a later build (518293 or later). Could you please ask your contact TCE to write me to prepare a proper build?

BR,

Vladimir

Milian_W_ · ‎06-28-2017

Thanks Vladimir, I have now opened another ticket for that.

Cheers

Milian_W_ · ‎07-03-2017

Collection now works fine with Update 4 (build 518293), but finalizing the data fails on my Arch system:

Data collection is completed successfully
Mon 03 Jul 2017 10:06:30 AM CEST The result file '/home/milian/intel/amplxe/projects/lab_mandelbrot_concurrent/r007cc/r007cc.amplxe' is successfully created and added to the project lab_mandelbrot_concurrent.

Failed to finalize the result
Mon 03 Jul 2017 10:06:31 AM CEST Cannot finalize the result.

Error 0x40000003 (Unexpected internal error / invalid state) -- locale::facet::_S_create_c_locale name not valid

Opening the data with an older VTune release finalizes it properly and I can open it. Opening the now-finalized data in the newer VTune also shows me that this update seems to require an update to my license/license server. So overall this is far from pretty...

Milian_W_ · ‎07-03-2017

On an Ubuntu system it works a charm! Even with the 4.10 kernel.

Thanks

Vladimir_R_Intel · ‎07-03-2017

Hi Milian,

Could you please upload your result? We'll look at the finalization issue.

When you wrote 'on my Arch system', do you mean Arch Linux? Could you please write an output of uname -a?

Also do you see any issues with license on an Ubuntu system?

BR,

Vladimir

Milian_W_ · ‎07-03-2017

On Ubuntu, I'm using a single-user license, not a floating user license that goes through our FlexLM. Thus licensing works fine there. In the past I also had to update the FlexLM server / license every now and then when a new VTune version was released. I suspect the same is true for Update 4 but to my knowledge there is no new FlexLM version available yet?

And yes, the arch system runs ArchLinux with 4.9.34-1-lts. The error sounds locale related, but that one looks fine to me:

$ locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=

I'll see if I can upload the result later.

Kelly_K_ · ‎08-08-2017

I am experiencing the same error when using PIN. Can anyone confirm if there will be a new PIN release?

Zhao__Ethan · ‎11-17-2019

I got the same error when working on amplxe-cl collect

/opt/intel/vtune_amplifier/bin64/amplxe-cl -collect locksandwaits -target-pid=9929

amplxe: Warning: The analysis type 'locksandwaits' is deprecated. Use 'threading' analysis type instead. See more details with 'amplxe-cl -help collect threading'.
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /data1/sunkaicheng/indigo/brec_indigo_recall/build/r002lw -command stop.
amplxe: Warning: Cannot stop posix timer: __NR_timer_settime() system call returned -1.
amplxe: Collection detached.
amplxe: Collection stopped.
amplxe: Using result path `/data1/sunkaicheng/indigo/brec_indigo_recall/build/r002lw'
amplxe: Executing actions 50 % done
amplxe: Error: 0x40000003 (Unexpected internal error / invalid state) -- locale::facet::_S_create_c_locale name not valid

Zhao__Ethan · ‎11-17-2019

I got the same error

/opt/intel/vtune_amplifier/bin64/amplxe-cl -collect locksandwaits -target-pid=9929

amplxe: Warning: The analysis type 'locksandwaits' is deprecated. Use 'threading' analysis type instead. See more details with 'amplxe-cl -help collect threading'.
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /data1/sunkaicheng/indigo/brec_indigo_recall/build/r002lw -command stop.
amplxe: Warning: Cannot stop posix timer: __NR_timer_settime() system call returned -1.
amplxe: Collection detached.
amplxe: Collection stopped.
amplxe: Using result path `/data1/sunkaicheng/indigo/brec_indigo_recall/build/r002lw'
amplxe: Executing actions 50 % done
amplxe: Error: 0x40000003 (Unexpected internal error / invalid state) -- locale::facet::_S_create_c_locale name not valid