Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Zheng__Eric
Beginner
132 Views

VTune 2018 outputs "out of memory" when memory is far away of exhaustion

Hello,

I want to analyze hotspots for a program but vtune2018 outputs "out of memory" and some assertion fail(after a period time of execution). But actually, memory is far away of exhaustion. Its output is as following

Freepages: Out of memory
vcs/collectunits1/sal/src/linux/sal.c:961 sal_munmap: Assertion 'result == 0' failed.
 
vcs/collectunits1/sal/src/linux/sal.c:961 sal_munmap: Assertion 'result == 0' failed.
 
AMPLXE_TPSSCOLLECTOR: init:1785: ((((&(g_tpss_module_map->cache_list))->prev == (&(g_tpss_module_map->cache_list))) && ((&(g_tpss_module_map->cache_list))->next == (&(g_tpss_module_map->cache_list))))) : 
Assertion failed: init:1785: ((((&(g_tpss_module_map->cache_list))->prev == (&(g_tpss_module_map->cache_list))) && ((&(g_tpss_module_map->cache_list))->next == (&(g_tpss_module_map->cache_list))))) : . Please contact the technical support. amplxe: Error: Assertion failed: ?invalid?. Please contact the technical support. 
amplxe: Collection stopped.

And if I execute it with srun in another machine(with the same environment and hardware), vtune says it starts but no corresponding process is created(only amplex-cl process exists). Could anyone help me with this problem?

My environment is 

  • Ubuntu 16.04.4 LTS
  • Intel(R) VTune(TM) Amplifier 2018 Update 2 (build 551022) Command Line Tool
  • Selftest reuslt
    Intel(R) VTune(TM) Amplifier Self Check Utility
    Copyright (C) 2009-2017 Intel Corporation. All rights reserved.
    Build Number: 551022
 
    Instrumentation based analysis check
    Example of analysis types: Hotspots, Concurrency, Locks and Waits
        Collection: Ok
        Finalization: Ok
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libpthread.so.0'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libc.so.6'.
    amplxe: Warning: Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.2.0.551022/lib64/libtpsstool.so'.
        Report: Ok
 
    HW event-based analysis check (Intel driver)
    Example of analysis types: Advanced Hotspots, HPC Performance Characterization, etc.
        Collection: Ok
    amplxe: Warning: To enable hardware event-base sampling, VTune Amplifier has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.
        Finalization: Ok
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/ld-2.23.so'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libc-2.23.so'.
    amplxe: Warning: Function and source-level analysis for the Linux kernel will not be possible since neither debug version of the kernel nor kernel symbol tables are found. See the Enabling Linux Kernel An
    alysis topic in the product online help for instructions.
        Report: Ok
 
    HW event-based analysis check (Intel driver)
    Example of analysis types: General Exploration
        Collection: Ok
    amplxe: Warning: To enable hardware event-base sampling, VTune Amplifier has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.
        Finalization: Ok
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/ld-2.23.so'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/modules/4.4.0-116-generic/kernel/drivers/net/ethernet/broadcom/tg3.ko'.
    amplxe: Warning: Function and source-level analysis for the Linux kernel will not be possible since neither debug version of the kernel nor kernel symbol tables are found. See the Enabling Linux Kernel An
    alysis topic in the product online help for instructions.
        Report: Ok
 
    HW event-based analysis with uncore events (Intel driver)
    Example of analysis types: Memory Access
        Collection: Ok
    amplxe: Warning: To enable hardware event-base sampling, VTune Amplifier has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.
        Finalization: Ok
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libgcc_s.so.1'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libpthread-2.23.so'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/modules/4.4.0-116-generic/kernel/drivers/net/ethernet/broadcom/tg3.ko'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/modules/4.4.0-116-generic/kernel/arch/x86/kvm/kvm.ko'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libc-2.23.so'.
    amplxe: Warning: Function and source-level analysis for the Linux kernel will not be possible since neither debug version of the kernel nor kernel symbol tables are found. See the Enabling Linux Kernel An
    alysis topic in the product online help for instructions.
        Report: Ok
 
    HW event-based analysis with stacks (Intel driver)
    Example of analysis types: Advanced Hotspots with Stacks, etc.
        Collection: Ok
    amplxe: Warning: To enable hardware event-base sampling, VTune Amplifier has disabled the NMI watchdog timer. The watchdog timer will be re-enabled after collection completes.
        Finalization: Ok
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libgcc_s.so.1'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libpthread-2.23.so'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/ld-2.23.so'.
    amplxe: Warning: Cannot locate debugging information for file `/lib/x86_64-linux-gnu/libm-2.23.so'.
 
    amplxe: Warning: Function and source-level analysis for the Linux kernel will not be possible since neither debug version of the kernel nor kernel symbol tables are found. See the Enabling Linux Kernel An
    alysis topic in the product online help for instructions.
        Report: Ok
 
    The system is ready to be used for performance analysis with Intel VTune Amplifier.
    Review warnings in the output above to find product limitations, if any.

This program is open source, if necessary I can provide it.

Thank you in advance.

0 Kudos
1 Reply
Vladimir_R_Intel
Employee
132 Views

Hi Eric,

So as the self check has passed it seems on some specific interaction with this particular application, so please share it.

And you can use advanced hotspots with stacks analysis as a workaround.

BR,

Vladimir

Reply