Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2154 Discussions

MPI_Init crash after setting I_MPI_STATS

AndrewC
New Contributor III
708 Views

After setting I_MPI_STATS=4, Intel MPI 4.1.3 on Linux crashes at MPI_Init

*** glibc detected ***  free(): invalid pointer: 0x0000000002933384 ***
======= Backtrace: =========
/lib64/libc.so.6[0x3193e7164f]
/lib64/libc.so.6(cfree+0x4b)[0x3193e7587b]
[0x231e318]
MPIR_Init_thread+0x62d)[0x231f7bd]
(MPI_Init+0x218)[0x2313598]

 

 

 

0 Kudos
9 Replies
AndrewC
New Contributor III
708 Views

Interestingly, I found that by setting I_MPI_STATS_FILE the crash went away...

0 Kudos
James_T_Intel
Moderator
707 Views

Very odd.  Are you doing anything with stdout or stderr?

0 Kudos
AndrewC
New Contributor III
708 Views

Nothing special with stdout/stderr. The code being executed is your typical  FORTRAN executable.

By setting I_MPI_STATS_FILE everything works as expected.

0 Kudos
James_T_Intel
Moderator
708 Views

What do you have I_MPI_STATS_BUCKETS set to?

0 Kudos
AndrewC
New Contributor III
708 Views

It was not set. But this crash is happening at MPI_Init

0 Kudos
AndrewC
New Contributor III
708 Views

What I mean is, no statistics have been gathered , so I do not see how any other I_MPI_STATS_XXXX settings would affect the issue.

0 Kudos
James_T_Intel
Moderator
708 Views

If I_MPI_STATS_BUCKETS is not set, I_MPI_STATS=4 shouldn't give anything.  I'm not able to reproduce the problem, is it only with a specific code, or with any code?

0 Kudos
AndrewC
New Contributor III
708 Views

That is not the case, I only set I_MPI_STATS and I_MPI_STATS_FILE , and I get meaningful information in the file as expected.

I don't have any other MPI software to test with, so cannot comment.

Here is the I_MPI_XXX dump.

[0] MPI startup(): I_MPI_DEBUG=100
[0] MPI startup(): I_MPI_FABRICS=shm:tcp
[0] MPI startup(): I_MPI_INFO_BRAND=Intel(R) Xeon(R)
[0] MPI startup(): I_MPI_INFO_CACHE1=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE2=0,1,2,3
[0] MPI startup(): I_MPI_INFO_CACHE3=0,0,0,0
[0] MPI startup(): I_MPI_INFO_CACHES=3
[0] MPI startup(): I_MPI_INFO_CACHE_SHARE=2,2,16
[0] MPI startup(): I_MPI_INFO_CACHE_SIZE=32768,262144,6291456
[0] MPI startup(): I_MPI_INFO_CORE=0,1,2,3
[0] MPI startup(): I_MPI_INFO_C_NAME=Wolfdale
[0] MPI startup(): I_MPI_INFO_DESC=1342208505
[0] MPI startup(): I_MPI_INFO_FLGB=0
[0] MPI startup(): I_MPI_INFO_FLGC=398124031
[0] MPI startup(): I_MPI_INFO_FLGD=-1075053569
[0] MPI startup(): I_MPI_INFO_LCPU=4
[0] MPI startup(): I_MPI_INFO_MODE=263
[0] MPI startup(): I_MPI_INFO_PACK=0,0,0,0
[0] MPI startup(): I_MPI_INFO_SERIAL=E31225
[0] MPI startup(): I_MPI_INFO_SIGN=132775
[0] MPI startup(): I_MPI_INFO_STATE=0
[0] MPI startup(): I_MPI_INFO_THREAD=0,0,0,0
[0] MPI startup(): I_MPI_INFO_VEND=1
[0] MPI startup(): I_MPI_PIN_INFO=x0,1,2,3
[0] MPI startup(): I_MPI_PIN_MAPPING=1:0 0
[0] MPI startup(): I_MPI_STATS=4
[0] MPI startup(): I_MPI_STATS_FILE=/home/andrewc/mpi_stats.txt

 

0 Kudos
James_T_Intel
Moderator
708 Views

Can you send me the code?  Maybe there is something specific to the code causing the problem.

0 Kudos
Reply