topic What I mean is, no statistics in Intel® MPI Library

MPI_Init crash after setting I_MPI_STATS

AndrewC — Fri, 31 Jan 2014 00:50:39 GMT

After setting I_MPI_STATS=4, Intel MPI 4.1.3 on Linux crashes at MPI_Init

*** glibc detected *** free(): invalid pointer: 0x0000000002933384 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3193e7164f]
/lib64/libc.so.6(cfree+0x4b)[0x3193e7587b]
[0x231e318]
MPIR_Init_thread+0x62d)[0x231f7bd]
(MPI_Init+0x218)[0x2313598]

Interestingly, I found that

AndrewC — Fri, 31 Jan 2014 01:23:09 GMT

Interestingly, I found that by setting I_MPI_STATS_FILE the crash went away...

Very odd. Are you doing

James_T_Intel — Fri, 31 Jan 2014 20:26:50 GMT

Very odd. Are you doing anything with stdout or stderr?

Nothing special with stdout

AndrewC — Fri, 31 Jan 2014 20:44:46 GMT

Nothing special with stdout/stderr. The code being executed is your typical FORTRAN executable.

By setting I_MPI_STATS_FILE everything works as expected.

What do you have I_MPI_STATUS

James_T_Intel — Fri, 31 Jan 2014 20:49:00 GMT

What do you have I_MPI_STATS_BUCKETS set to?

It was not set. But this

AndrewC — Fri, 31 Jan 2014 21:10:36 GMT

It was not set. But this crash is happening at MPI_Init

What I mean is, no statistics

AndrewC — Fri, 31 Jan 2014 21:28:48 GMT

What I mean is, no statistics have been gathered , so I do not see how any other I_MPI_STATS_XXXX settings would affect the issue.

If I_MPI_STATS_BUCKETS is not

James_T_Intel — Fri, 31 Jan 2014 21:45:47 GMT

If I_MPI_STATS_BUCKETS is not set, I_MPI_STATS=4 shouldn't give anything. I'm not able to reproduce the problem, is it only with a specific code, or with any code?

That is not the case, I only

AndrewC — Fri, 31 Jan 2014 22:03:47 GMT

That is not the case, I only set I_MPI_STATS and I_MPI_STATS_FILE , and I get meaningful information in the file as expected.

I don't have any other MPI software to test with, so cannot comment.

Here is the I_MPI_XXX dump.

[0] MPI startup(): I_MPI_DEBUG=100
[0] MPI startup(): I_MPI_FABRICS=shm:tcp
[0] MPI startup(): I_MPI_INFO_BRAND=Intel(R) Xeon(R)
[0] MPI startup(): I_MPI_INFO_CACHE1=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE2=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE3=0,0,0,0
[0] MPI startup(): I_MPI_INFO_CACHES=3
[0] MPI startup(): I_MPI_INFO_CACHE_SHARE=2,2,16
[0] MPI startup(): I_MPI_INFO_CACHE_SIZE=32768,262144,6291456
[0] MPI startup(): I_MPI_INFO_CORE=0,1,2,3
[0] MPI startup(): I_MPI_INFO_C_NAME=Wolfdale
[0] MPI startup(): I_MPI_INFO_DESC=1342208505
[0] MPI startup(): I_MPI_INFO_FLGB=0
[0] MPI startup(): I_MPI_INFO_FLGC=398124031
[0] MPI startup(): I_MPI_INFO_FLGD=-1075053569
[0] MPI startup(): I_MPI_INFO_LCPU=4
[0] MPI startup(): I_MPI_INFO_MODE=263
[0] MPI startup(): I_MPI_INFO_PACK=0,0,0,0
[0] MPI startup(): I_MPI_INFO_SERIAL=E31225
[0] MPI startup(): I_MPI_INFO_SIGN=132775
[0] MPI startup(): I_MPI_INFO_STATE=0
[0] MPI startup(): I_MPI_INFO_THREAD=0,0,0,0
[0] MPI startup(): I_MPI_INFO_VEND=1
[0] MPI startup(): I_MPI_PIN_INFO=x0,1,2,3
[0] MPI startup(): I_MPI_PIN_MAPPING=1:0 0
[0] MPI startup(): I_MPI_STATS=4
[0] MPI startup(): I_MPI_STATS_FILE=/home/andrewc/mpi_stats.txt

Can you send me the code?

James_T_Intel — Fri, 07 Feb 2014 19:06:39 GMT

Can you send me the code? Maybe there is something specific to the code causing the problem.