Problems getting remote sampling to work with VTune Amplifier 2019

[Leigh__Richard]

I have recently tried using VTune Amplifier 2019 to perform analysis of an application using the remote Linux SSH capability. When trying to set up e.g. Hotspot analysis for the remote target, I get the following errors:

• This analysis type is not applicable to the system because VTune Amplifier cannot recognize the processor. If this is a new Intel processor, please check for an updated version of VTune Amplifier. If this is an unreleased Intel processor, please contact Online Service Cetner for an NDA product package.
• This analysis type is not applicable to the current machine microarchitecture.
• Cannot enable Hardware Event-based Sampling due to a problem with the driver (sep*/sepdrv*). Check that the driver is running and the driver group is in the current user group list. See the "Sampling Drivers" help topic for further details.
• Cannot enable advanced capabilities for Hardware Event-based Sampling due to a problem with the driver (vtss/vtssp). Check that the driver is running and the driver group is in the current user group list. See the "Sampling Drivers" help topic for further details.

The remote system was setup by following the instructions here https://software.intel.com/en-us/vtune-amplifier-install-guide-linux-installing-command-line-collectors and adding my username to the 'vtune' group after having set up passwordless SSH access between the two. The remote system is an Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz system running Centos7.5, and the output of

/opt/intel/vtune_amplifier_2019/bin64/amplxe-self-checker.sh

on both machines warns about some (to my mind) uninteresting issues:

amplxe: Warning: Cannot locate debugging information for file `/lib64/libc.so.6'.

amplxe: Warning: Cannot locate debugging information for file `/lib64/libpthread.so.0'.

amplxe: Warning: To enable hardware event-base sampling, VTune Amplifier has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.

amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libc-2.17.so'.

amplxe: Warning: Function and source-level analysis for the Linux kernel will not be possible since neither debug version of the kernel nor kernel symbol tables are found. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.

and finishes with the line

The system is ready to be used for performance analysis with Intel VTune Amplifier.

The output of lsmod | egrep 'vtss|pax|sep' is:

vtsspp                384857  0

sep5                  856640  0

socperf3               33124  1 sep5

pax                    13820  0

This is using the vtune_amplifier_2019.0.2.570779 package.

Does anybody know what I could be doing wrong here?

[Vitaly_S_Intel]

Hi Richard,

Can you please provide output of the following command on the target system:

> <VTune install path>/bin64/sep -version

[Leigh__Richard]

/opt/intel/vtune_amplifier_2019/bin64/sep -version

Sampling Enabling Product version: 5.1 Beta built on Aug  8 2018 02:28:49

SEP User Mode Version: 5.1.0 Beta

SEP Driver Version: 5.1.0 Beta

PAX Driver Version: 1.0.2

Platform type: 108

CPU name: Intel(R) Xeon(R) Processor code named Broadwell

PMU: broadwell_server

Driver configs: Non-Maskable Interrupt

Copyright(C) 2007-2018 Intel Corporation. All rights reserved.

[Leigh__Richard · Accepted solution]

I finally figured this out. Our sysadmins configured SSH to print various banners and log in message in the default system set up. This was clearly confusing vtune when it was trying to parse command results issued over SSH as this text was always being sent first.

With a combination of creating a .hushlogin file in my home directory on the remote side, and setting LogLevel Error in my .ssh/config on the GUI side, I am able to do remote analysis properly.

Hope this can help anyone else seeing this problem!

[Vitaly_S_Intel]

Thanks Richard for your own triaging and providing hints how to solve this. We'll also consider proper handling of welcome message in VTune remote communication.