Analyzers
Talk to fellow users of Intel Analyzer tools (Intel VTune™ Profiler, Intel Advisor)

Vtune runtime failure

dkokron
Beginner
754 Views

Vtune fails with the following after the application has finished. 

"vcs/collectunits1/tmu/src/tmu.c:421 write_trace: Assertion 'Cannot write magic record to trace' failed."

I am running an OpenFOAM application across 20 nodes with a total of 800 cores.  My launch command is

mpiexec -np 800 vtune -collect memory-access -r MemAcc.${NCPUS}p.${CPU}.${PBS_JOBID} -finalization-mode=full -data-limit=8192 sonicFoam -parallel

 

OpenFOAM is compiled and run under the following environment

module purge

source /nas/sw/sdev/intel/oneapi/2022/v2/compiler/latest/env/vars.sh
source /nas/sw/sdev/intel/oneapi/2022/v2/mkl/latest/env/vars.sh
source /nas/sw/sdev/intel/oneapi/2021/v2/mpi/latest/env/vars.sh
source /nas/sw/sdev/intel/oneapi/2022/v2/vtune/latest/env/vars.sh

 

I have run the vtune-self-checker.sh on all of the nodes associated with the above failure.  Many though not all of the nodes hang on the "context switches" test.

 

Some system info:

Intel(R) VTune(TM) Profiler 2022.2.0 (build 623516) Command Line Tool

NAME="Red Hat Enterprise Linux Server"
VERSION="7.6 (Maipo)"

Processors: 2 x 40 Intel(R) Xeon(R) Gold 6242R CPU @ 3.10GHz

0 Kudos
3 Replies
VaradJ_Intel
Moderator
697 Views

Hi,

 

Good day to you.

 

Thanks for posting in Intel Communities.

 

We are extremely sorry that the Intel VTune Profiler is causing issues with your workflow.

 

To clearly assist you with your issue, can you share the output of the following commands:

 

$ sudo <vtune-install-dir>/sepdk/src/insmod-sep -r

$ sudo <vtune-install-dir>/sepdk/src/insmod-sep -q

$ sudo <vtune-install-dir>/bin64/vtune-self-checker.sh

 

Eg.:

sudo /opt/intel/oneapi/vtune/latest/sepdk/src/insmod-sep -r

sudo /opt/intel/oneapi/vtune/latest/sepdk/src/insmod-sep -q

sudo /opt/intel/oneapi/vtune/latest/bin64/vtune-self-checker.sh


 

To replicate the issue from our side, can you share a sample reproducer code along with the exact steps or commands that you followed to do the analysis.


Thank You.


0 Kudos
VaradJ_Intel
Moderator
669 Views

Hi,


We have not heard back from you. Please could you give us an update?


Thank You.


0 Kudos
VaradJ_Intel
Moderator
649 Views

Hi,


We assume that your issue is resolved. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.


Thank You


0 Kudos
Reply