topic ITAC: profiling without tracing in Intel® MPI Library

ITAC: profiling without tracing

davvad — Tue, 07 Feb 2012 06:58:08 GMT

Hello,

I have the following question. Trace Collector provides very large data for my app, so I want just to collect profile information, without trace data.
As I understood from Collector Reference Guide (paragraph 3.6, "Recording Statistical Information") it is possible by setting enviromental variables VT_STATISTICS=ON and VT_PROCESS=OFF.

But in this case my app (that runs for 1.5 minutes on 16 processes) gathers 1.6 GB of data...

What can be the problem? Why trace data is all the same being collected?

Thanks in advance!

ITAC: profiling without tracing

Dmitry_K_Intel2 — Tue, 07 Feb 2012 07:49:04 GMT

Hi Davvad,

I've glanced at the issue and probably VT_PROCESS format should be slightly different.
You can try to use a configuration file like:
$less vt.conf
# enable statistics gathering
STATISTICS ON
# no need to gather trace data
PROCESS 0:N OFF
LOGFILE-FORMAT STFSINGLE
$ export VT_CONFIC=./vt.conf
$mpiexec -trace -n 4 IMB-MPI1

Also why not to try I_MPI_STATS option?
You don't need to recompile or rebuild your application. Just run it with "-genv I_MPI_STATS N" where N from 1 to 10. And you can set to "ipm" - you'll get an output which looks like theIPMformat.

BTW: Using of a configuration file you can gather only those functions which are critical for your application.

Regards!
Dmitry

ITAC: profiling without tracing

davvad — Tue, 07 Feb 2012 13:38:21 GMT

Hello Dmirty,

Thanks for your quick response!

I've tried what you ecommended, but all is the same..

But I've noticed that despite the fact that I'm using conf file, *.prot file created by ITC contains following:
# LOGFILE-FORMAT was not set explicitly.
LOGFILE-FORMAT "STF"
...
# STATISTICS was not set explicitly.
STATISTICS 0
...

My conf file contains:
# enable statistics gathering
STATISTICS ON
# # no need to gather trace data
PROCESS 0:N OFF
VERBOSE 2
LOGFILE-FORMAT STFSINGLE

And VT_CONFIG variable is set:
[vadim@t60-2 Konraz_ITAC]$ echo $VT_CONFIG
/home/vadim/Konraz_ITAC/itac.conf

Does it means that for some reason configure options wasn't received by ITC?

P.S. In case this can help to solve this problem, this is make/run commands I use:
Compile:
mpicxx -O3 -DUSE_MPI -c Konraz27.cpp

Link:
mpicxx Konraz27.o -L$VT_LIB_DIR -lVT $VT_ADD_LIBS -o Konraz27.run

Run:
mpirun -np 4 -q hdd -maxtime 15 ./Konraz27.run

ITAC: profiling without tracing

Dmitry_K_Intel2 — Tue, 07 Feb 2012 14:04:32 GMT

Vadim,

Could you please try the following:
mpicxx -O3 -DUSE_MPI -o Konraz27.runKonraz27.cpp
mpirun -trace -np 4 -q hdd -maxtime 15 ./Konraz27.run
In this case TraceCollector library will be attached dynamically.

Also, it would be nice to know versions of MPI and TraceCollector (Analyzer) you are using.

Regards!
Dmitry

ITAC: profiling without tracing

davvad — Tue, 07 Feb 2012 14:40:36 GMT

Dmirty,

Unfortanutely, -trace option isn't recognized by mpirun command...

Version of ITAC: 8.0.0.011
Version of MPI (hope I got it right): mpich 1.2.7

ITAC: profiling without tracing

Dmitry_K_Intel2 — Wed, 08 Feb 2012 11:05:31 GMT

Right, OK!
Vadim, I'm not sure that working with MPICH ITAC can support all options. You will be able to gather traces but I cannot promise functionality will be supported in full.
Ideally, you need to use Intel MPI library and mpirun (mpiexec) shipped with this library.

Also, Intel Trace Analyzer and Collector version 8.0 Update 3 is available and if you have rights you can download it from premier.intel.com. This is the version I had checked functionality with vt.conf file.

Regards!
Dmitry

ITAC: profiling without tracing

davvad — Wed, 08 Feb 2012 12:14:28 GMT

Dmitry,

Thanks for your advice!
I 've used Intel MPI and that helped! Now config file is being read and I get only stats, no trace data.

Is it possible to view stats in Trace Analyzer or should I use stftool only?