VTune Amplifier XE 2017 (3.0.510739) on Linux no longer invoking child application

[Andrew_J_1]

Hello, I recently started evaluating VTune internally, and I had some good successes initially. However, I was working with the VTune "scripting" team on trying to diagnose a Python issue with VTune, and now VTune has just "given up". It is worth noting that, at the current time, the Python analysis only supports the "basic" hotspot (and Locks-n-Waits) collectors. To elaborate on my issue: whatever kind of binary is passed as follows: * amplxe-cl -c hotspots nothing is collected, and all that is reported is: * amplxe: Error: 0x40000025 (Inapplicable report) -- The report 'summary' is not applicable to the result However, I fully understand this error when nothing is collected/the process is too short-lived. The problem here is that VTune is simply not invoking/launching the child job -- the process I wish to analyse has print statements (for example), and these _do not_ appear. This is why there is no data collector, the child process isn't being launched at all. VTune did previously work on my machine before I updated my Linux kernel (I updated from 3.10.0-514.21.1.el7.x86_64 to 3.10.0-514.21.2.el7.x86_64 - the smallest of point releases!). After I noticed that VTune was broken, I even completely uninstalled/reinstalled the product, but to no avail. Other things I have tried: * advanced-hotspots _do_ work, but these are incompatible with Python analyses (so they do not relevant for the purposes of this post) * manually making sure the kernel modules are loaded using insmod-sep but these have also made no difference! I was extremely impressed with what VTune could do before it "broke", so this is extremely infuriating! Any helps/tips/suggestions/things I can try to give you all more information? Cheers, Andrew

[Andrew_J_1]

Attached to this reply is: * The contents of the directory "/tmp/amplxe_failures", populated by this command "export AMPLXE_LOG_DIR=/tmp/amplxe_failures/amplxe_log; export AMPLXE_LOG_LEVEL=TRACE; rm -rf /tmp/amplxe_failures/r000ah; amplxe-cl -collect hotspots -target-tmp-dir=/tmp/amplxe_failures -r /tmp/amplxe_failures/r000ah -- $(which python) test.py" * The output of amplxe-feedback -create-bug-report report.zip

[Vladimir_R_Intel]

Hi Andrew,

Could you please try to check the following commands?

1. amplxe-cl -c hotspots /bin/ls 

2. amplxe-cl -c hotspots -mrte-mode=native <your_application_with_parameters>

3. pin -xyzzy -mesgon log_injector -mesgon log_syscall -unique_logfile -- <your_application_with_parameters>

And if the third item will fail please send me collected logs.

BR,

Vladimir

[Andrew_J_1]

Hello Vladimir,

Please find the output of the below script attached to this reply:

#!/bin/bash -x

LOG_DIR=amplxe_log
APP="/usr/bin/python test.py"

rm -rf r0*
rm -rf ${LOG_DIR}
rm -rf *.txt
rm -rf pin.*
mkdir ${LOG_DIR}
export AMPLXE_LOG_DIR=$(pwd)/${LOG_DIR}
export AMPLXE_LOG_LEVEL=TRACE

amplxe-cl -c hotspots /bin/ls |& cat > ls_output.txt
amplxe-cl -c hotspots -mrte-mode=native $APP |& cat > native_app_output.txt
pin -xyzzy -mesgon log_injector -mesgon log_syscall -unique_logfile -- $APP |& cat > pin_output.txt

Interestingly, when I run:

pin -xyzzy -mesgon log_injector -mesgon log_syscall -unique_logfile -- /usr/bin/python test.py

I get the print out of:

Bus error

on my terminal. As such, I ran the above command through GDB, and here is the back-trace:

#0  0x00007ffff7ee02eb in OS_BARESYSCALL_DoCallAsmIntel64Linux () from /opt/intel/vtune_amplifier_xe_2017.3.0.510739/lib64/pinruntime/libc-dynamic.so
#1  0x00007fffffffcd50 in ?? ()
#2  0x00007ffff7ee0424 in BARESYSCALL_Do () from /opt/intel/vtune_amplifier_xe_2017.3.0.510739/lib64/pinruntime/libc-dynamic.so
#3  0x00007ffff7eded36 in OS_SendSignalToThread () from /opt/intel/vtune_amplifier_xe_2017.3.0.510739/lib64/pinruntime/libc-dynamic.so
#4  0x00007ffff7edd8fd in OS_RaiseException () from /opt/intel/vtune_amplifier_xe_2017.3.0.510739/lib64/pinruntime/libc-dynamic.so
#5  0x00007ffff7e8c25a in raise () from /opt/intel/vtune_amplifier_xe_2017.3.0.510739/lib64/pinruntime/libc-dynamic.so
#6  0x00005555558cd0ce in ?? ()
#7  0x00005555558cd1ce in LEVEL_INJECTOR::DoSystemChecks() ()
#8  0x00005555558c1b5e in LEVEL_INJECTOR::UNIX_INJECTOR::Run() ()
#9  0x00005555558c6905 in LEVEL_INJECTOR::PIN_UNIX_ENVIRONMENT::LaunchPin() ()
#10 0x00005555558ae79d in LEVEL_INJECTOR::PIN_ENVIRONMENT::Main() ()
#11 0x0000555555644029 in main ()

I should note that I was previously able to run VTune, but since doing a package upgrade on RHEL, I am susequently unable to run the "basic" hotspots (advanced hotspots still run!). As I mentioned in my earlier response, this problem has persisted even after reinstalling VTune.

Cheers,

Andrew

[Vladimir_R_Intel]

Andrew,

we suppose that this issue is fixed in the upcoming VTune AXE 2017 U4.

Please make additional actions to file Online Service Center (https://supporttickets.intel.com/?lang=en-US) issue with request for trying updated version of VTune with a fix (you need to refer on this forum thread in OSC request).

In this case we will be able to provide you a pre-release VTune package.

[Andrew_J_1]

Hi Vladimir,

I actually signed-up for the 2018 "beta" for VTune (version 2018.0.023) -- might these fixes be in 2017 U4?

Cheers,

Andrew

[Vladimir_R_Intel]

Hi Andrew,

2018 Beta doesn't include these fixes.

VTune 2017 U4 will be the first release with them and then they will be included in 2018 Gold too.

BR,

Vladimir

[Andrew_J_1]

Hello Vladimir,

I am currently unable to request support for VTune, due to a website issue. I have raised this with another part of Intel that I cannot access the support area.

As soon as that issue is resolved, I will look at getting 2017 U4 of VTune to see if this resolves my issue.

Thanks for your help so far -- cheers,

Andrew

[Andrew_J_1]

Hello Vladimir, ​Okay, I now have access to the support area. I have raised this issue as request number 02920984. Cheers, Andrew

[Andrew_J_1]

Hello Vladimir, Unfortunately, because I am currently only an eval customer, Intel's support is unable to give me access to the preview of U4. However, in the mean time, I downgraded my Linux kernel from 3.10.0-514.21.2.el7.x86_64 to 3.10.0-514.21.1.el7.x86_64 (as before, the tiniest of version changes!), and now VTune works again. I have been informed that U4 will be officially released around July 10th; when it comes out, I will check to confirm if this problem persists. Cheers, Andrew

[Andrew_J_1]

I presume that this issue is related to: https://bugs.centos.org/view.php?id=13453