Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)
4995 Discussions

zero CPU usage with vtune on Xeon Platinum 8160

AR1
Beginner
‎07-09-2018 12:12 PM
456 Views

Hi,

I'm trying to analyze the performance of a hybrid MPI & OpenMP code using intel vtune. I'm compiling my code using intel/18.0.2 & impi/18.0.2 & vtune/18.0.2 The OS is rhel 3.10.0-693.17.1.el7.x86_64 and CPU's are Intel Xeon Platinum 8160 (Skylake). I compile my code using "mpicc -xCORE-AVX512 -qopenmp mycode.c -g -O3" (I tried -O0 but didn't help), and I submit the code using "amplxe-cl -collect hpc-performance -r r000hpc ./a.out"

The code doesn't run at all and just jumps out (without vtune it runs without any problem). I get the following error message:

amplxe: Peak bandwidth measurement started.
amplxe: Peak bandwidth measurement finished.
amplxe: Collection started. To stop the collection, either press CTRL-C or enter from another console window: amplxe-cl -r /work/vtune/r000hpc -command stop.
amplxe: Collection stopped.
amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 19 % Resolving information for `libittnotify_collecto
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libnss_files-2.17.so'.
amplxe: Executing actions 19 % Resolving information for `libpsm2.so.2.1'      
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/ld-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.2.0.551022/lib64/runtime/libittnotify_collector.so'.
amplxe: Executing actions 19 % Resolving information for `libc-2.17.so'        
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libpsm2.so.2.1'.
amplxe: Warning: Cannot locate file `igb.ko'.
amplxe: Executing actions 19 % Resolving information for `igb'                 
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libc-2.17.so'.
amplxe: Executing actions 21 % Resolving information for `vmlinux'             
amplxe: Warning: Cannot locate file `dm_mod.ko'.
amplxe: Executing actions 21 % Resolving information for `dm_mod'              
amplxe: Warning: Cannot locate file `sep4_1.ko'.
amplxe: Executing actions 22 % Resolving information for `sep4_1'              
amplxe: Warning: Cannot locate debugging information for the Linux kernel. Source-level analysis will not be possible. Function-level analysis will be limited to kernel symbol tables. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.
amplxe: Executing actions 100 % done

and when I use "amplxe-cl -report summary -r r000hpc -report-knob show-issues=false" I get a zero usage result like this:

amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 75 % Generating a report                             Elapsed Time: 0.355s
SP GFLOPS: 0.000
Effective Physical Core Utilization: 0.3% (0.157 out of 48)
    Effective Logical Core Utilization: 0.2% (0.209 out of 96)
Memory Bound: 1.2% of Pipeline Slots
    Cache Bound: 9.9% of Clockticks
    DRAM Bound: 0.0% of Clockticks
        DRAM Bandwidth Bound: 0.0% of Elapsed Time
    NUMA: % of Remote Accesses: 0.0%
FPU Utilization: 0.0%
    SP FLOPs per Cycle: 0.000 Out of 64
    Vector Capacity Usage: 0.0%
    FP Instruction Mix
        % of Packed FP Instr.: 0.0%
            % of 128-bit: 0.0%
            % of 256-bit: 0.0%
            % of 512-bit: 0.0%
        % of Scalar FP Instr.: 0.0%
    FP Arith/Mem Rd Instr. Ratio: 0.000
    FP Arith/Mem Wr Instr. Ratio: 0.000
Collection and Platform Info
    Application Command Line: ./a.out
    User Name: ######## (I erased it)
    Operating System: 3.10.0-693.17.1.el7.x86_64 NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"  CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
    Computer Name: ######### (I erased it)
    Result Size: 6 MB
    Collection start time: 18:36:37 09/07/2018 UTC
    Collection stop time: 18:36:38 09/07/2018 UTC
    Collector Type: Event-based sampling driver
    CPU
        Name: Intel(R) Xeon(R) Processor code named Skylake
        Frequency: 2.095 GHz
        Logical CPU Count: 96
amplxe: Executing actions 100 % done

Do you know what I am doing wrong?

Thank you very much!

AR

0 Kudos

Link Copied

3 Replies
Vladimir_R_Intel
Employee
‎07-10-2018 01:21 AM
456 Views

Hi,

please send the whole command line you used to run the collection. Also it will be helpful if you can set an absolute path to your application, repeat your experiment and upload here a failed result.

BR,

Vladimir

0 Kudos
Copy link
AR1
Beginner
‎07-10-2018 09:26 AM
456 Views

Hi Vladimir,

I compile the code using "mpicc -xCORE-AVX512 -qopenmp mycode.c -g -O3"

I submit mycode using a batch script, with this line inside the script: "ibrun amplxe-cl -collect hpc-performance -r /work/vtune/r000hpc ./a.out"

and I run vtune afterwards, using "amplxe-cl -report summary -r /work/vtune/r000hpc/ -report-knob show-issues=false"

The result looks like:

amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 75 % Generating a report                             Elapsed Time: 0.287s
SP GFLOPS: 0.000
Effective Physical Core Utilization: 0.3% (0.129 out of 48)
    Effective Logical Core Utilization: 0.3% (0.259 out of 96)
Memory Bound: 0.0% of Pipeline Slots
    Cache Bound
    DRAM Bound
        DRAM Bandwidth Bound: 0.0% of Elapsed Time
    NUMA: % of Remote Accesses: 0.0%
FPU Utilization: 0.0%
    SP FLOPs per Cycle: 0.000 Out of 64
    Vector Capacity Usage: 0.0%
    FP Instruction Mix
        % of Packed FP Instr.: 0.0%
            % of 128-bit: 0.0%
            % of 256-bit: 0.0%
            % of 512-bit: 0.0%
        % of Scalar FP Instr.: 0.0%
    FP Arith/Mem Rd Instr. Ratio: 0.000
    FP Arith/Mem Wr Instr. Ratio: 0.000
Collection and Platform Info
    Application Command Line: ./a.out
    User Name: ########(I erased this)
    Operating System: 3.10.0-693.17.1.el7.x86_64 NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/"  CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7"
    Computer Name: ##########(I erased this)
    Result Size: 5 MB
    Collection start time: 16:13:33 10/07/2018 UTC
    Collection stop time: 16:13:33 10/07/2018 UTC
    Collector Type: Event-based sampling driver
    CPU
        Name: Intel(R) Xeon(R) Processor code named Skylake
        Frequency: 2.095 GHz
        Logical CPU Count: 96

 

 

Here is the error file:

Currently Loaded Modules:
  1) git/2.9.0       3) xalt/2.0.7   5) intel/18.0.2      7) impi/18.0.2
  2) autotools/1.1   4) ####(I erased this)         6) libfabric/1.6.1   8) vtune/18.0.2

Inactive Modules:
  1) python

amplxe: Analyzing data in the node-wide mode. The hostname (######) will be added to the result path/name.
amplxe: Peak bandwidth measurement started.
amplxe: Peak bandwidth measurement finished.
amplxe: Collection started.
amplxe: Collection stopped.
amplxe: Using result path `/work/vtune/r000hpc'
amplxe: Executing actions 19 % Resolving information for `libc-2.17.so'        
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libnss_files-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/ld-2.17.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libpsm2.so.2.1'.
amplxe: Warning: Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.2.0.551022/lib64/runtime/libittnotify_collector.so'.
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/libc-2.17.so'.
amplxe: Warning: Cannot locate file `dm_mod.ko'.
amplxe: Executing actions 20 % Resolving information for `librdmacm.so.1.0.13'
amplxe: Warning: Cannot locate debugging information for file `/usr/lib64/librdmacm.so.1.0.13'.
amplxe: Executing actions 20 % Resolving information for `libmpi.so.12.0'      
amplxe: Warning: Cannot locate file `hfi1.ko'.
amplxe: Executing actions 21 % Resolving information for `libiomp5.so'         
amplxe: Warning: Cannot locate file `xfs.ko'.
amplxe: Executing actions 22 % Resolving information for `xfs'                 
amplxe: Warning: Cannot locate debugging information for the Linux kernel. Source-level analysis will not be possible. Function-level analysis will be limited to kernel symbol tables. See the Enabling Linux Kernel Analysis topic in the product online help for instructions.
amplxe: Executing actions 100 % done

 

What are these warnings about?

 

0 Kudos
Copy link
Vladimir_R_Intel
Employee
‎07-11-2018 01:19 AM
456 Views

Hi AR,

This is resolving warnings that means that a collector gathered some info but the tool cannot find symbol information for a particular module to attribute info to some function. So as you see these warnings and collection has ended I can make a conclusion that VTune has launched something that ended very quickly.

Currently I cannot say why it is so.  I think that I can say more based on the failed result, under the failed result I mean the r000hpc or any other result collected by VTune in this scenario. And as I said above it will be helpful if you change your script to use a full absolute path to your application.

But if it isn't possible we can try few additional checks to localize the problem. Please try the following:

1. try to collect hotspots and advanced-hotspots results

2. try to analyze another existing standard binary, for example ls or printenv. It will be something like ibrun amplxe-cl -collect hpc-performance -r /work/vtune/r001hpc  printenv

3. try to collect something without ibrun

BR,

Vladimir

 

 

0 Kudos
Copy link
Reply

Community support is provided during standard business hours (Monday to Friday 7AM - 5PM PST). Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.