VTune and Application slowdown/hang

[Marc_C_]

Hi Folks,

I'm working on profiling a third-party database server running on a single baremetal Skylake Xeon Platinum 8180 node running CentOS 7.2.  The server software is running as root user as well as the vtune command-line tool.  When using any preconfigured user sampling profile such as hotspots, VTune will not stop recording after the given duration.  In addition, my query benchmark running on a separate client machine runs well past its own time limits on execution.  At a certain point, the query rate just slows to a crawl and I cannot break out of amplxe-cl using Ctrl-C or its own stop command (that gives me an internal error).  Any advice is much appreciated.

-Marc

UPDATE:  Through some strange combination of *when* I start the benchmark, I'm able to capture some of the trace inside the GUI.  It finalizes results but the collection log shows:

Data collection is completed with warnings
    Tue 19 Dec 2017 12:51:18 AM CET  The result file '/root/profiling/vtune/ML/MarkLogic/r003hs/r003hs.amplxe' is created and added to the project MarkLogic. Please see warning messages for details. 
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.
    Cannot stop posix timer: __NR_timer_settime() system call returned -1.

Finalization completed with warnings 
    Tue 19 Dec 2017 12:52:20 AM CET  Result finalization has completed with warnings that may affect the representation of the analysis data. Please see details below. 
    The database has been cleared, elapsed time is 1.096 seconds.
    Raw data has been loaded to the database, elapsed time is 14.503 seconds.
    Data transformations have been finished, elapsed time is 0.019 seconds.
    Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.1.0.535340/lib64/pinruntime/libc-dynamic.so'.
    XXXredactedXXXX
    Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.1.0.535340/bin64/pinbin'.
    Cannot locate debugging information for file `/opt/intel/vtune_amplifier_2018.1.0.535340/lib64/libtpsstool.so'.
    Symbol resolution has been finished, elapsed time is 36.520 seconds.
    Deferred data transformations have been finished, elapsed time is 1.452 seconds.
    Data model parameters have been set, elapsed time is 0.096 seconds.
    Precomputing frequently used data has been finished, elapsed time is 0.913 seconds.
    Redundant overtime data has been discarded, elapsed time is 0.005 seconds.
    Raw collector data has been discarded, elapsed time is 0.000 seconds.
    Finalizing the result took 57.474 seconds.

[Vladimir_R_Intel]

Hi Marc,

I need to clarify few things. As I understood you try an attach to a third-party database server.

Does the issue reproduce only with the server? Or it doesn't matter which application you attach to? Or an attach is not the case at all and the issue reproduces with every application regardless of collection flow?

Could you please upload result of failed collection and the "successful" result with warnings?

please describe what your server is. I mean the language it is based on, the rate of signals it's used, or maybe it works inside a container and so on. Any details will be very appreciated.

Also please try to use advanced-hotspots as a possible workaround.

BR,

Vladimir