Heap-buffer-overflow in MPI_Init

asd__asdqwe · ‎08-15-2016

Hello,

I'm compiling this example:

#include <mpi.h>
#include <stdio.h>

int main(int argc, char** argv) {
    MPI_Init(NULL, NULL);
    MPI_Finalize();
}

And I get the following memory error. Is this a false positive? Could this be fixed if not?

$  mpicxx -show && mpicxx hello.cpp -fsanitize=address && mpirun -np 1 ./a.out
g++ -I/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/include -L/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt -L/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.1/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.1/intel64/lib -lmpicxx -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread
=================================================================
==12984==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61200000b0c4 at pc 0x7fe42bb4e93c bp 0x7ffec5b32310 sp 0x7ffec5b31ac0
READ of size 261 at 0x61200000b0c4 thread T0
    #0 0x7fe42bb4e93b  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x4393b)
    #1 0x7fe42bb4ed27 in __interceptor_strstr (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x43d27)
    #2 0x7fe42b0431b4 in MPID_nem_impi_configuration_init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x2c51b4)
    #3 0x7fe42b0459f0 in MPID_nem_init_ckpt (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x2c79f0)
    #4 0x7fe42ae8d340 in MPIDI_CH3_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x10f340)
    #5 0x7fe42b035b7e in MPID_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x2b7b7e)
    #6 0x7fe42afe5364 in MPIR_Init_thread (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x267364)
    #7 0x7fe42afd671d in MPI_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x25871d)
    #8 0x400873 in main (/tmp/a.out+0x400873)
    #9 0x7fe429b3872f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2072f)
    #10 0x400788 in _start (/tmp/a.out+0x400788)

0x61200000b0c4 is located 0 bytes to the right of 260-byte region [0x61200000afc0,0x61200000b0c4)
allocated by thread T0 here:
    #0 0x7fe42bbcce38 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1e38)
    #1 0x7fe42b033a98  (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/libmpi.so.12+0x2b5a98)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x4393b) 
Shadow bytes around the buggy address:
  0x0c247fff95c0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
  0x0c247fff95d0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c247fff95e0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fa
  0x0c247fff95f0: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c247fff9600: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c247fff9610: 00 00 00 00 00 00 00 00[04]fa fa fa fa fa fa fa
  0x0c247fff9620: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c247fff9630: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x0c247fff9640: 00 00 00 00 00 00 00 00 01 fa fa fa fa fa fa fa
  0x0c247fff9650: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
  0x0c247fff9660: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==12984==ABORTING

For reference, here is what I get with MPICH:

$ mpicxx.mpich hello.cpp -fsanitize=address && mpirun.mpich -np 1 ./a.out && echo $?
0

Thank you for your help.

Dmitry_S_Intel · ‎08-15-2016

Hi,

Thank you for the message.

Can you try with debug libraries?

For using debug you need to source

<installdir>/intel64em64t/bin/mpivars.sh debug_mt

--

Dmitry

asd__asdqwe · ‎08-15-2016

Here is what I get now:

$ mpicxx -show && mpicxx -g hello.cpp -fsanitize=address && mpirun -np 1 ./a.out 
g++ -I/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/include -L/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt -L/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib -Xlinker --enable-new-dtags -Xlinker -rpath -Xlinker /opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.1/intel64/lib/release_mt -Xlinker -rpath -Xlinker /opt/intel/mpi-rt/5.1/intel64/lib -lmpicxx -lmpifort -lmpi -lmpigi -ldl -lrt -lpthread
=================================================================
==25225==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000ebb1 at pc 0x7fd06d3f6019 bp 0x7ffed9a0c410 sp 0x7ffed9a0bbc0
READ of size 1 at 0x60200000ebb1 thread T0
    #0 0x7fd06d3f6018  (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x51018)
    #1 0x7fd06c72bc94  (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x43fc94)
    #2 0x7fd06c72f4e7 in MPID_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x4434e7)
    #3 0x7fd06c6afaeb in MPIR_Init_thread (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x3c3aeb)
    #4 0x7fd06c6988bf in MPI_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x3ac8bf)
    #5 0x400863 in main /tmp/hello.cpp:5
    #6 0x7fd06b0a672f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2072f)
    #7 0x400778 in _start (/tmp/a.out+0x400778)

0x60200000ebb1 is located 0 bytes to the right of 1-byte region [0x60200000ebb0,0x60200000ebb1)
allocated by thread T0 here:
    #0 0x7fd06d466e38 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.3+0xc1e38)
    #1 0x7fd06c72b90c  (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x43f90c)
    #2 0x7fd06c72f4e7 in MPID_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x4434e7)
    #3 0x7fd06c6afaeb in MPIR_Init_thread (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x3c3aeb)
    #4 0x7fd06c6988bf in MPI_Init (/opt/intel/compilers_and_libraries_2016.2.181/linux/mpi/intel64/lib/debug_mt/libmpi.so.12+0x3ac8bf)
    #5 0x400863 in main /tmp/hello.cpp:5
    #6 0x7fd06b0a672f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2072f)

SUMMARY: AddressSanitizer: heap-buffer-overflow (/usr/lib/x86_64-linux-gnu/libasan.so.3+0x51018) 
Shadow bytes around the buggy address:
  0x0c047fff9d20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9d30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9d40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9d50: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c047fff9d60: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c047fff9d70: fa fa 03 fa fa fa[01]fa fa fa 02 fa fa fa 07 fa
  0x0c047fff9d80: fa fa 00 fa fa fa 04 fa fa fa fd fd fa fa 00 fa
  0x0c047fff9d90: fa fa 04 fa fa fa 00 04 fa fa 00 04 fa fa fd fa
  0x0c047fff9da0: fa fa 00 fa fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff9db0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa fd fd
  0x0c047fff9dc0: fa fa fd fd fa fa fd fd fa fa fd fd fa fa 07 fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Heap right redzone:      fb
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack partial redzone:   f4
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==25225==ABORTING

Dmitry_S_Intel · ‎08-15-2016

Thank you.

We have some reports about potential memory leak in MPI_Init. Our investigations indicated that they are false positive.

I've submit the ticket against this issue to the development team, and let you know results.

--

Dmitry

asd__asdqwe · ‎08-16-2016

Alright, thank you for your help. Please keep me posted. Thank you.

Dmitry_S_Intel · ‎09-06-2016

Hi,

The development team investigated this issue and confirmed that it is already reported false-positive.

Thank you for your investigations.

--

Dmitry