Intel® HPC Toolkit
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2146 Discussions

MPI program memory error: corrupted double-linked list: 0x00000000017f3f10

Qiyu8
Beginner
551 Views

I'm using Intel oneAPI 2021 compiling my MPI program, This is the make.rule:

CC_OPTIMIZATIONS =	-O3 
FC_OPTIMIZATIONS =	-O3 
SFC_OPTIMIZATIONS =	-O3 

####################################################

CXX		=       mpiicpc
CXXFLAGS	=       $(CXX_US_FLAGS) $(CXX_INCLUDES) $(CXX_COMPILE_OPTIONS) #$(SMP_FLAGS)

CC		=       mpiicc
CFLAGS		=       $(CC_US_FLAGS) $(CC_INCLUDES) $(CC_COMPILE_OPTIONS) $(CC_OPTIMIZATIONS) $(DMP_FLAGS) $(SMP_FLAGS)

FC		=       mpiifort
FFLAGS		=       $(FC_US_FLAGS) $(FC_INCLUDES) $(FC_COMPILE_OPTIONS) $(FC_OPTIMIZATIONS)

SFC		=       mpiifort
SFFLAGS		=       $(FC_US_FLAGS) $(FC_INCLUDES) $(SFC_COMPILE_OPTIONS) $(SFC_OPTIMIZATIONS)

LDFLAGS		=       $(CC_COMPILE_OPTIONS) $(SFC_OPTIMIZATIONS) 

when running with :

mpirun -hostfile $HOSTFILE -np 200 -ppn 100 mpi_program.exe
 
The error occurred:
*** Error in `./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe': corrupted double-linked list: 0x00000000014daea0 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7f3e4)[0x2b6037c3c3e4]
/lib64/libc.so.6(+0x82ba8)[0x2b6037c3fba8]
/lib64/libc.so.6(__libc_malloc+0x4c)[0x2b6037c426fc]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0xa6272c]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0xa61292]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0xa5d771]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0xa5a0e7]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0xa22253]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0x411931]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0x4118dd]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x2b6037bdf555]
./vdras.inst-3km.start-2022-09-04_03:00:00_UTC.exe[0x4117f6]
======= Memory map: ========
00400000-00c78000 r-xp 00000000 00:33 617957                             /v2020/main/rmapsnow-all.exe
00e77000-00e79000 r--p 00877000 00:33 617957                             /v2020/main/rmapsnow-all.exe
00e79000-00e84000 rw-p 00879000 00:33 617957                             /v2020/main/rmapsnow-all.exe
00e84000-00f4d000 rw-p 00000000 00:00 0 
013ba000-0141a000 rw-p 00000000 00:00 0                                  [heap]
0141a000-0141b000 rw-p 00000000 00:00 0                                  [heap]
0141b000-01452000 rw-p 00000000 00:00 0                                  [heap]
01452000-0145a000 rw-p 00000000 00:00 0                                  [heap]
0145a000-01476000 rw-p 00000000 00:00 0                                  [heap]
01476000-0147e000 rw-p 00000000 00:00 0                                  [heap]
0147e000-01535000 rw-p 00000000 00:00 0                                  [heap]
2b6035e3b000-2b6035e5d000 r-xp 00000000 08:03 787239                     /usr/lib64/ld-2.17.so
2b6035e5d000-2b6035e69000 rw-p 00000000 00:00 0 
2b6035e69000-2b6035e6a000 rw-s 00000000 00:13 394439631                  /dev/shm/Intel_MPI_SO1Aq9 (deleted)
2b6035e6a000-2b6035e6b000 -w-s 00000000 00:05 11430                      /dev/infiniband/uverbs0
2b6035e6b000-2b6035e6c000 -w-s 00001000 00:05 11430                      /dev/infiniband/uverbs0
2b6035e6c000-2b6035e70000 rw-p 00000000 00:00 0 
2b6035e70000-2b6035e7a000 r--p 00000000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e7a000-2b6035e8c000 r-xp 0000a000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e8c000-2b6035e92000 r--p 0001c000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e92000-2b6035e93000 ---p 00022000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e93000-2b6035e95000 r--p 00022000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e95000-2b6035e96000 rw-p 00024000 00:33 522257                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifport.so.5
2b6035e96000-2b6035e99000 rw-p 00000000 00:00 0 
2b6035e99000-2b6035eb9000 r--p 00000000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035eb9000-2b6035f87000 r-xp 00020000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035f87000-2b6035fb0000 r--p 000ee000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035fb0000-2b6035fb1000 ---p 00117000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035fb1000-2b6035fb6000 r--p 00117000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035fb6000-2b6035fb7000 rw-p 0011c000 00:33 547774                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libifcoremt.so.5
2b6035fb7000-2b6036018000 rw-p 00000000 00:00 0 
2b6036018000-2b6036019000 -w-s 00002000 00:05 11430                      /dev/infiniband/uverbs0
2b6036019000-2b603601a000 -w-s 00003000 00:05 11430                      /dev/infiniband/uverbs0
2b603601a000-2b603601b000 -w-s 00004000 00:05 11430                      /dev/infiniband/uverbs0
2b603601b000-2b603601c000 -w-s 00005000 00:05 11430                      /dev/infiniband/uverbs0
2b603601c000-2b603601d000 -w-s 00006000 00:05 11430                      /dev/infiniband/uverbs0
2b603601d000-2b603601e000 -w-s 00007000 00:05 11430                      /dev/infiniband/uverbs0
2b603601e000-2b603601f000 r--s 00500000 00:05 11430                      /dev/infiniband/uverbs0
2b603601f000-2b6036020000 r--s 00700000 00:05 11430                      /dev/infiniband/uverbs0
2b6036020000-2b6036021000 rw-s 00800000 00:05 11430                      /dev/infiniband/uverbs0
2b6036021000-2b6036022000 -w-s 00600000 00:05 11430                      /dev/infiniband/uverbs0
2b6036022000-2b6036058000 rw-p 00000000 00:00 0 
2b603605a000-2b603605c000 rw-p 00000000 00:00 0 
2b603605c000-2b603605d000 r--p 00021000 08:03 787239                     /usr/lib64/ld-2.17.so
2b603605d000-2b603605e000 rw-p 00022000 08:03 787239                     /usr/lib64/ld-2.17.so
2b603605e000-2b603605f000 rw-p 00000000 00:00 0 
2b603605f000-2b603610e000 r-xp 00000000 00:33 701112                     /opt/software/netcdf/4.7.4/lib/libnetcdff.so.7.0.0
2b603610e000-2b603630d000 ---p 000af000 00:33 701112                     /opt/software/netcdf/4.7.4/lib/libnetcdff.so.7.0.0
2b603630d000-2b603630e000 r--p 000ae000 00:33 701112                     /opt/software/netcdf/4.7.4/lib/libnetcdff.so.7.0.0
2b603630e000-2b603630f000 rw-p 000af000 00:33 701112                     /opt/software/netcdf/4.7.4/lib/libnetcdff.so.7.0.0
2b603630f000-2b60364e9000 rw-p 00000000 00:00 0 
2b60364e9000-2b603678a000 r-xp 00000000 00:33 701107                     /opt/software/netcdf/4.7.4/lib/libnetcdf.so.18.0.0
2b603678a000-2b603698a000 ---p 002a1000 00:33 701107                     /opt/software/netcdf/4.7.4/lib/libnetcdf.so.18.0.0
2b603698a000-2b603698f000 r--p 002a1000 00:33 701107                     /opt/software/netcdf/4.7.4/lib/libnetcdf.so.18.0.0
2b603698f000-2b6036998000 rw-p 002a6000 00:33 701107                     /opt/software/netcdf/4.7.4/lib/libnetcdf.so.18.0.0
2b6036998000-2b603699a000 rw-p 00000000 00:00 0 
2b603699a000-2b60369bc000 r-xp 00000000 00:33 724045                     /opt/software/hdf5/1.8.12/lib/libhdf5_hl.so.8.0.1
2b60369bc000-2b6036bbb000 ---p 00022000 00:33 724045                     /opt/software/hdf5/1.8.12/lib/libhdf5_hl.so.8.0.1
2b6036bbb000-2b6036bbc000 r--p 00021000 00:33 724045                     /opt/software/hdf5/1.8.12/lib/libhdf5_hl.so.8.0.1
2b6036bbc000-2b6036bbd000 rw-p 00022000 00:33 724045                     /opt/software/hdf5/1.8.12/lib/libhdf5_hl.so.8.0.1
2b6036bbd000-2b6036bce000 rw-p 00000000 00:00 0 
2b6036bce000-2b6036f78000 r-xp 00000000 00:33 660262                     /opt/software/hdf5/1.8.12/lib/libhdf5.so.8.0.1
2b6036f78000-2b6037177000 ---p 003aa000 00:33 660262                     /opt/software/hdf5/1.8.12/lib/libhdf5.so.8.0.1
2b6037177000-2b603717d000 r--p 003a9000 00:33 660262                     /opt/software/hdf5/1.8.12/lib/libhdf5.so.8.0.1
2b603717d000-2b6037183000 rw-p 003af000 00:33 660262                     /opt/software/hdf5/1.8.12/lib/libhdf5.so.8.0.1
2b6037183000-2b6037185000 rw-p 00000000 00:00 0 
2b6037185000-2b603719d000 r-xp 00000000 00:33 466426                     /opt/xc/installcode/zlib-1.2.7/lib/libz.so.1.2.7
2b603719d000-2b603739c000 ---p 00018000 00:33 466426                     /opt/xc/installcode/zlib-1.2.7/lib/libz.so.1.2.7
2b603739c000-2b603739d000 r--p 00017000 00:33 466426                     /opt/xc/installcode/zlib-1.2.7/lib/libz.so.1.2.7
2b603739d000-2b603739e000 rw-p 00018000 00:33 466426                     /opt/xc/installcode/zlib-1.2.7/lib/libz.so.1.2.7
2b603739e000-2b6037487000 r-xp 00000000 08:03 787295                     /usr/lib64/libstdc++.so.6.0.19
2b6037487000-2b6037686000 ---p 000e9000 08:03 787295                     /usr/lib64/libstdc++.so.6.0.19
2b6037686000-2b603768e000 r--p 000e8000 08:03 787295                     /usr/lib64/libstdc++.so.6.0.19
2b603768e000-2b6037690000 rw-p 000f0000 08:03 787295                     /usr/lib64/libstdc++.so.6.0.19
2b6037690000-2b60376a5000 rw-p 00000000 00:00 0 
2b60376a5000-2b60376ba000 r-xp 00000000 08:03 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2b60376ba000-2b60378b9000 ---p 00015000 08:03 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2b60378b9000-2b60378ba000 r--p 00014000 08:03 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2b60378ba000-2b60378bb000 rw-p 00015000 08:03 786451                     /usr/lib64/libgcc_s-4.8.5-20150702.so.1
2b60378bb000-2b60379bc000 r-xp 00000000 08:03 787254                     /usr/lib64/libm-2.17.so
2b60379bc000-2b6037bbb000 ---p 00101000 08:03 787254                     /usr/lib64/libm-2.17.so
2b6037bbb000-2b6037bbc000 r--p 00100000 08:03 787254                     /usr/lib64/libm-2.17.so
2b6037bbc000-2b6037bbd000 rw-p 00101000 08:03 787254                     /usr/lib64/libm-2.17.so
2b6037bbd000-2b6037cb2000 r-xp 00000000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cb2000-2b6037cb3000 r-xp 000f5000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cb3000-2b6037cb5000 r-xp 000f6000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cb5000-2b6037cb6000 r-xp 000f8000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cb6000-2b6037cbc000 r-xp 000f9000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cbc000-2b6037cbe000 r-xp 000ff000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037cbe000-2b6037d80000 r-xp 00101000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037d80000-2b6037f80000 ---p 001c3000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037f80000-2b6037f84000 r--p 001c3000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037f84000-2b6037f86000 rw-p 001c7000 08:03 787246                     /usr/lib64/libc-2.17.so
2b6037f86000-2b6037f8b000 rw-p 00000000 00:00 0 
2b6037f8b000-2b6038113000 r-xp 00000000 00:33 415368                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpifort.so.12.0.0
2b6038113000-2b6038313000 ---p 00188000 00:33 415368                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpifort.so.12.0.0
2b6038313000-2b6038317000 r--p 00188000 00:33 415368                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpifort.so.12.0.0
2b6038317000-2b603831b000 rw-p 0018c000 00:33 415368                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpifort.so.12.0.0
2b603831b000-2b603833f000 rw-p 00000000 00:00 0 
2b603833f000-2b6039169000 r-xp 00000000 00:33 612067                     /opt/intel/oneapi/mpi/2021.7.0/lib/release/libmpi.so.12.0.0
2b6039169000-2b6039369000 ---p 00e2a000 00:33 612067                     /opt/intel/oneapi/mpi/2021.7.0/lib/release/libmpi.so.12.0.0
2b6039369000-2b603937a000 r--p 00e2a000 00:33 612067                     /opt/intel/oneapi/mpi/2021.7.0/lib/release/libmpi.so.12.0.0
2b603937a000-2b603938e000 rw-p 00e3b000 00:33 612067                     /opt/intel/oneapi/mpi/2021.7.0/lib/release/libmpi.so.12.0.0
2b603938e000-2b6039b74000 rw-p 00000000 00:00 0 
2b6039b74000-2b6039b76000 r-xp 00000000 08:03 787252                     /usr/lib64/libdl-2.17.so
2b6039b76000-2b6039d76000 ---p 00002000 08:03 787252                     /usr/lib64/libdl-2.17.so
2b6039d76000-2b6039d77000 r--p 00002000 08:03 787252                     /usr/lib64/libdl-2.17.so
2b6039d77000-2b6039d78000 rw-p 00003000 08:03 787252                     /usr/lib64/libdl-2.17.so
2b6039d78000-2b6039d7f000 r-xp 00000000 08:03 787276                     /usr/lib64/librt-2.17.so
2b6039d7f000-2b6039f7e000 ---p 00007000 08:03 787276                     /usr/lib64/librt-2.17.so
2b6039f7e000-2b6039f7f000 r--p 00006000 08:03 787276                     /usr/lib64/librt-2.17.so
2b6039f7f000-2b6039f80000 rw-p 00007000 08:03 787276                     /usr/lib64/librt-2.17.so
2b6039f80000-2b6039f97000 r-xp 00000000 08:03 787272                     /usr/lib64/libpthread-2.17.so
2b6039f97000-2b603a196000 ---p 00017000 08:03 787272                     /usr/lib64/libpthread-2.17.so
2b603a196000-2b603a197000 r--p 00016000 08:03 787272                     /usr/lib64/libpthread-2.17.so
2b603a197000-2b603a198000 rw-p 00017000 08:03 787272                     /usr/lib64/libpthread-2.17.so
2b603a198000-2b603a19c000 rw-p 00000000 00:00 0 
2b603a19c000-2b603ab11000 r-xp 00000000 00:33 421641                     /opt/software/pnetcdf/1.11.2/lib/libpnetcdf.so.3.0.2
2b603ab11000-2b603ad11000 ---p 00975000 00:33 421641                     /opt/software/pnetcdf/1.11.2/lib/libpnetcdf.so.3.0.2
2b603ad11000-2b603ad13000 r--p 00975000 00:33 421641                     /opt/software/pnetcdf/1.11.2/lib/libpnetcdf.so.3.0.2
2b603ad13000-2b603ad20000 rw-p 00977000 00:33 421641                     /opt/software/pnetcdf/1.11.2/lib/libpnetcdf.so.3.0.2
2b603ad20000-2b603ad2c000 rw-p 00000000 00:00 0 
2b603ad2c000-2b603ad33000 r--p 00000000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad33000-2b603ad90000 r-xp 00007000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad90000-2b603ad9a000 r--p 00064000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad9a000-2b603ad9b000 ---p 0006e000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad9b000-2b603ad9c000 r--p 0006e000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad9c000-2b603ad9d000 rw-p 0006f000 00:33 366141                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirc.so
2b603ad9d000-2b603ad9e000 rw-p 00000000 00:00 0 
2b603ad9e000-2b603adbb000 r--p 00000000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603adbb000-2b603af2d000 r-xp 0001d000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603af2d000-2b603b14b000 r--p 0018f000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603b14b000-2b603b14c000 ---p 003ad000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603b14c000-2b603b152000 r--p 003ad000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603b152000-2b603b196000 rw-p 003b3000 00:33 521518                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libimf.so
2b603b196000-2b603b333000 r--p 00000000 00:33 366142                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libsvml.so
2b603b333000-2b603bd5a000 r-xp 0019d000 00:33 366142                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libsvml.so
2b603bd5a000-2b603c6ae000 r--p 00bc4000 00:33 366142                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libsvml.so
2b603c6ae000-2b603c74e000 r--p 01517000 00:33 366142                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libsvml.so
2b603c74e000-2b603c7a7000 rw-p 015b7000 00:33 366142                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libsvml.so
2b603c7a7000-2b603c7ae000 r--p 00000000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c7ae000-2b603c80b000 r-xp 00007000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c80b000-2b603c815000 r--p 00064000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c815000-2b603c816000 ---p 0006e000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c816000-2b603c817000 r--p 0006e000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c817000-2b603c818000 rw-p 0006f000 00:33 521519                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libintlc.so.5
2b603c818000-2b603c819000 rw-p 00000000 00:00 0 
2b603c819000-2b603c970000 r-xp 00000000 00:33 611331                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirng.so
2b603c970000-2b603cb6f000 ---p 00157000 00:33 611331                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirng.so
2b603cb6f000-2b603cb70000 r--p 00156000 00:33 611331                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirng.so
2b603cb70000-2b603cb83000 rw-p 00157000 00:33 611331                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libirng.so
2b603cb83000-2b603cba1000 r-xp 00000000 00:33 497836                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpicxx.so.12.0.0
2b603cba1000-2b603cda0000 ---p 0001e000 00:33 497836                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpicxx.so.12.0.0
2b603cda0000-2b603cda1000 r--p 0001d000 00:33 497836                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpicxx.so.12.0.0
2b603cda1000-2b603cda3000 rw-p 0001e000 00:33 497836                     /opt/intel/oneapi/mpi/2021.7.0/lib/libmpicxx.so.12.0.0
2b603cda3000-2b603cddb000 r-xp 00000000 00:33 489474                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5
2b603cddb000-2b603cfdb000 ---p 00038000 00:33 489474                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5
2b603cfdb000-2b603cfdc000 r--p 00038000 00:33 489474                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5
2b603cfdc000-2b603cfdf000 rw-p 00039000 00:33 489474                     /opt/intel/oneapi/compiler/2022.2.0/linux/compiler/lib/intel64_lin/libcilkrts.so.5
2b603cfdf000-2b603cfe0000 rw-p 00000000 00:00 0 
2b603cfe0000-2b603cfea000 r-xp 00000000 08:03 793342                     /usr/lib64/libnuma.so.1.0.0
2b603cfea000-2b603d1ea000 ---p 0000a000 08:03 793342                     /usr/lib64/libnuma.so.1.0.0
2b603d1ea000-2b603d1eb000 r--p 0000a000 08:03 793342                     /usr/lib64/libnuma.so.1.0.0
2b603d1eb000-2b603d1ec000 rw-p 0000b000 08:03 793342                     /usr/lib64/libnuma.so.1.0.0
2b603d1ec000-2b61e6de3000 rw-s 00000000 00:13 394439632                  /dev/shm/Intel_MPI_hG3ze9 (deleted)
2b61e6de3000-2b61ed6d6000 rw-s 1a9bf7000 00:13 394439632                 /dev/shm/Intel_MPI_hG3ze9 (deleted)
2b61ed6d6000-2b629e87d000 rw-s 1b04ea000 00:13 394439632                 /dev/shm/Intel_MPI_hG3ze9 (deleted)
2b629e87d000-2b629e8c6000 r-xp 00000000 00:33 415369                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/libfabric.so.1
2b629e8c6000-2b629eac5000 ---p 00049000 00:33 415369                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/libfabric.so.1
2b629eac5000-2b629eac8000 rw-p 00048000 00:33 415369                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/libfabric.so.1
2b629eac8000-2b629eac9000 rw-p 00000000 00:00 0 
2b629eac9000-2b629eb20000 r-xp 00000000 00:33 554479                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libverbs-1.1-fi.so
2b629eb20000-2b629ed1f000 ---p 00057000 00:33 554479                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libverbs-1.1-fi.so
2b629ed1f000-2b629ed23000 rw-p 00056000 00:33 554479                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libverbs-1.1-fi.so
2b629ed23000-2b629ed3a000 r-xp 00000000 08:03 802965                     /usr/lib64/librdmacm.so.1.2.28.0
2b629ed3a000-2b629ef39000 ---p 00017000 08:03 802965                     /usr/lib64/librdmacm.so.1.2.28.0
2b629ef39000-2b629ef3a000 r--p 00016000 08:03 802965                     /usr/lib64/librdmacm.so.1.2.28.0
2b629ef3a000-2b629ef3b000 rw-p 00017000 08:03 802965                     /usr/lib64/librdmacm.so.1.2.28.0
2b629ef3b000-2b629ef3c000 rw-p 00000000 00:00 0 
2b629ef3c000-2b629ef56000 r-xp 00000000 08:03 802166                     /usr/lib64/libibverbs.so.1.8.28.0
2b629ef56000-2b629f155000 ---p 0001a000 08:03 802166                     /usr/lib64/libibverbs.so.1.8.28.0
2b629f155000-2b629f156000 r--p 00019000 08:03 802166                     /usr/lib64/libibverbs.so.1.8.28.0
2b629f156000-2b629f157000 rw-p 0001a000 08:03 802166                     /usr/lib64/libibverbs.so.1.8.28.0
2b629f157000-2b629f175000 r-xp 00000000 08:03 788066                     /usr/lib64/libnl-3.so.200.23.0
2b629f175000-2b629f375000 ---p 0001e000 08:03 788066                     /usr/lib64/libnl-3.so.200.23.0
2b629f375000-2b629f377000 r--p 0001e000 08:03 788066                     /usr/lib64/libnl-3.so.200.23.0
2b629f377000-2b629f378000 rw-p 00020000 08:03 788066                     /usr/lib64/libnl-3.so.200.23.0
2b629f378000-2b629f3dc000 r-xp 00000000 08:03 788074                     /usr/lib64/libnl-route-3.so.200.23.0
2b629f3dc000-2b629f5db000 ---p 00064000 08:03 788074                     /usr/lib64/libnl-route-3.so.200.23.0
2b629f5db000-2b629f5de000 r--p 00063000 08:03 788074                     /usr/lib64/libnl-route-3.so.200.23.0
2b629f5de000-2b629f5e3000 rw-p 00066000 08:03 788074                     /usr/lib64/libnl-route-3.so.200.23.0
2b629f5e3000-2b629f5e5000 rw-p 00000000 00:00 0 
2b629f5e5000-2b629f62a000 r-xp 00000000 00:33 554478                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libtcp-fi.so
2b629f62a000-2b629f82a000 ---p 00045000 00:33 554478                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libtcp-fi.so
2b629f82a000-2b629f82d000 rw-p 00045000 00:33 554478                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libtcp-fi.so
2b629f82d000-2b629f888000 r-xp 00000000 00:33 587641                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libsockets-fi.so
2b629f888000-2b629fa87000 ---p 0005b000 00:33 587641                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libsockets-fi.so
2b629fa87000-2b629fa8b000 rw-p 0005a000 00:33 587641                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libsockets-fi.so
2b629fa8b000-2b629fadb000 r-xp 00000000 00:33 637211                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libshm-fi.so
2b629fadb000-2b629fcda000 ---p 00050000 00:33 637211                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libshm-fi.so
2b629fcda000-2b629fcde000 rw-p 0004f000 00:33 637211                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libshm-fi.so
2b629fcde000-2b629fd32000 r-xp 00000000 00:33 398917                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/librxm-fi.so
2b629fd32000-2b629ff31000 ---p 00054000 00:33 398917                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/librxm-fi.so
2b629ff31000-2b629ff35000 rw-p 00053000 00:33 398917                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/librxm-fi.so
2b629ff35000-2b62a0083000 r-xp 00000000 00:33 637210                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libpsm3-fi.so
2b62a0083000-2b62a0282000 ---p 0014e000 00:33 637210                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libpsm3-fi.so
2b62a0282000-2b62a028a000 rw-p 0014d000 00:33 637210                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libpsm3-fi.so
2b62a028a000-2b62a02b2000 rw-p 00000000 00:00 0 
2b62a02b2000-2b62a02b6000 r-xp 00000000 08:03 787474                     /usr/lib64/libuuid.so.1.3.0
2b62a02b6000-2b62a04b5000 ---p 00004000 08:03 787474                     /usr/lib64/libuuid.so.1.3.0
2b62a04b5000-2b62a04b6000 r--p 00003000 08:03 787474                     /usr/lib64/libuuid.so.1.3.0
2b62a04b6000-2b62a04b7000 rw-p 00004000 08:03 787474                     /usr/lib64/libuuid.so.1.3.0
2b62a04b7000-2b62a04f8000 r-xp 00000000 00:33 530012                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libmlx-fi.so
2b62a04f8000-2b62a06f8000 ---p 00041000 00:33 530012                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libmlx-fi.so
2b62a06f8000-2b62a06fb000 rw-p 00041000 00:33 530012                     /opt/intel/oneapi/mpi/2021.7.0/libfabric/lib/prov/libmlx-fi.so
2b62a06fb000-2b62a0754000 r-xp 00000000 08:03 803625                     /usr/lib64/libucp.so.0.0.0
2b62a0754000-2b62a0953000 ---p 00059000 08:03 803625                     /usr/lib64/libucp.so.0.0.0
2b62a0953000-2b62a0954000 r--p 00058000 08:03 803625                     /usr/lib64/libucp.so.0.0.0
2b62a0954000-2b62a0957000 rw-p 00059000 08:03 803625                     /usr/lib64/libucp.so.0.0.0
2b62a0957000-2b62a097d000 r-xp 00000000 08:03 803629                     /usr/lib64/libuct.so.0.0.0
2b62a097d000-2b62a0b7d000 ---p 00026000 08:03 803629                     /usr/lib64/libuct.so.0.0.0
2b62a0b7d000-2b62a0b7e000 r--p 00026000 08:03 803629                     /usr/lib64/libuct.so.0.0.0
2b62a0b7e000-2b62a0b82000 rw-p 00027000 08:03 803629                     /usr/lib64/libuct.so.0.0.0
2b62a0b82000-2b62a0cc1000 r-xp 00000000 08:03 803627                     /usr/lib64/libucs.so.0.0.0
2b62a0cc1000-2b62a0ec1000 ---p 0013f000 08:03 803627                     /usr/lib64/libucs.so.0.0.0
2b62a0ec1000-2b62a0ed4000 r--p 0013f000 08:03 803627                     /usr/lib64/libucs.so.0.0.0
2b62a0ed4000-2b62a0edb000 rw-p 00152000 08:03 803627                     /usr/lib64/libucs.so.0.0.0
2b62a0edb000-2b62a0ee3000 rw-p 00000000 00:00 0 
2b62a0ee3000-2b62a0ef5000 r-xp 00000000 08:03 803623                     /usr/lib64/libucm.so.0.0.0
2b62a0ef5000-2b62a10f4000 ---p 00012000 08:03 803623                     /usr/lib64/libucm.so.0.0.0
2b62a10f4000-2b62a10f5000 r--p 00011000 08:03 803623                     /usr/lib64/libucm.so.0.0.0
2b62a10f5000-2b62a10f6000 rw-p 00012000 08:03 803623                     /usr/lib64/libucm.so.0.0.0
2b62a10f6000-2b62a10f7000 rw-p 00000000 00:00 0 
2b62a10f7000-2b62a1152000 r-xp 00000000 08:03 925471                     /usr/lib64/ucx/libuct_ib.so.0.0.0
2b62a1152000-2b62a1352000 ---p 0005b000 08:03 925471                     /usr/lib64/ucx/libuct_ib.so.0.0.0
2b62a1352000-2b62a1353000 r--p 0005b000 08:03 925471                     /usr/lib64/ucx/libuct_ib.so.0.0.0
2b62a1353000-2b62a1358000 rw-p 0005c000 08:03 925471                     /usr/lib64/ucx/libuct_ib.so.0.0.0
2b62a1358000-2b62a13a0000 r-xp 00000000 08:03 802963                     /usr/lib64/libmlx5.so.1.12.28.0
2b62a13a0000-2b62a15a0000 ---p 00048000 08:03 802963                     /usr/lib64/libmlx5.so.1.12.28.0
2b62a15a0000-2b62a15a1000 r--p 00048000 08:03 802963                     /usr/lib64/libmlx5.so.1.12.28.0
2b62a15a1000-2b62a15a2000 rw-p 00049000 08:03 802963                     /usr/lib64/libmlx5.so.1.12.28.0
2b62a15a2000-2b62a15a4000 rw-p 00000000 00:00 0 
2b62a15a4000-2b62a15af000 r-xp 00000000 08:03 925473                     /usr/lib64/ucx/libuct_rdmacm.so.0.0.0
2b62a15af000-2b62a17ae000 ---p 0000b000 08:03 925473                     /usr/lib64/ucx/libuct_rdmacm.so.0.0.0
2b62a17ae000-2b62a17af000 r--p 0000a000 08:03 925473                     /usr/lib64/ucx/libuct_rdmacm.so.0.0.0
2b62a17af000-2b62a17b0000 rw-p 0000b000 08:03 925473                     /usr/lib64/ucx/libuct_rdmacm.so.0.0.0
2b62a17b0000-2b62a17b3000 r-xp 00000000 08:03 925469                     /usr/lib64/ucx/libuct_cma.so.0.0.0
2b62a17b3000-2b62a19b3000 ---p 00003000 08:03 925469                     /usr/lib64/ucx/libuct_cma.so.0.0.0
2b62a19b3000-2b62a19b4000 r--p 00003000 08:03 925469                     /usr/lib64/ucx/libuct_cma.so.0.0.0
2b62a19b4000-2b62a19b5000 rw-p 00004000 08:03 925469                     /usr/lib64/ucx/libuct_cma.so.0.0.0
2b62a19b5000-2b62a19b9000 r-xp 00000000 08:03 925475                     /usr/lib64/ucx/libuct_knem.so.0.0.0
2b62a19b9000-2b62a1bb8000 ---p 00004000 08:03 925475                     /usr/lib64/ucx/libuct_knem.so.0.0.0
2b62a1bb8000-2b62a1bb9000 r--p 00003000 08:03 925475                     /usr/lib64/ucx/libuct_knem.so.0.0.0
2b62a1bb9000-2b62a1bba000 rw-p 00004000 08:03 925475                     /usr/lib64/ucx/libuct_knem.so.0.0.0
2b62a1bba000-2b62a1c45000 rw-p 00000000 00:00 0 
2b62a1c45000-2b62a1cc5000 rw-p 00000000 00:00 0 
2b62a1cc5000-2b62a1cc7000 rw-p 00000000 00:00 0 
2b62a1cc7000-2b62a1d07000 rw-p 00000000 00:00 0 
2b62a1d07000-2b62a1d32000 rw-p 00000000 00:00 0 
2b62a1d32000-2b62a1d72000 rw-p 00000000 00:00 0 
2b62a1d72000-2b62a1d74000 rw-p 00000000 00:00 0 
2b62a1d74000-2b62a1e74000 rw-p 00000000 00:00 0 
2b62a1e74000-2b62a1e76000 rw-p 00000000 00:00 0 
2b62a1e76000-2b62a1ef6000 rw-p 00000000 00:00 0 
2b62a1ef6000-2b62a1f21000 rw-p 00000000 00:00 0 
2b62a1f21000-2b62a1f61000 rw-p 00000000 00:00 0 
2b62a1f61000-2b62a1f63000 rw-p 00000000 00:00 0 
2b62a1f63000-2b62a1fe3000 rw-p 00000000 00:00 0 
2b62a1fe3000-2b62a1fe5000 rw-p 00000000 00:00 0 
2b62a1fe5000-2b62a2005000 rw-p 00000000 00:00 0 
2b62a2005000-2b62a2006000 rw-p 00000000 00:00 0 
2b62a2006000-2b62a208b000 rw-p 00000000 00:00 0 
2b62a208b000-2b62a208c000 ---p 00000000 00:00 0 
2b62a208c000-2b62a228d000 rw-p 00000000 00:00 0 
2b62a228d000-2b62a230d000 rw-p 00000000 00:00 0 
2b62a230d000-2b62a230f000 rw-p 00000000 00:00 0 
2b62a230f000-2b62a232f000 rw-p 00000000 00:00 0 
2b62a232f000-2b62a2330000 rw-p 00000000 00:00 0 
2b62a2330000-2b62a23b5000 rw-p 00000000 00:00 0 
2b62a23e8000-2b62a2600000 rw-p 00000000 00:00 0 
2b62a2600000-2b62a2c00000 rw-p 00000000 00:00 0 
2b62a2c00000-2b62a2e00000 rw-p 00000000 00:00 0 
2b62a2e00000-2b62a5400000 rw-p 00000000 00:00 0 
2b62a5400000-2b62a7000000 rw-p 00000000 00:00 0 
2b62a7000000-2b62a7a00000 rw-p 00000000 00:00 0 
2b62a7a00000-2b62a7abb000 rw-p 00000000 00:00 0 
2b62a8000000-2b62a8200000 rw-p 00000000 00:00 0 
2b62a8200000-2b62a9600000 rw-p 00000000 00:00 0 
2b62a9600000-2b62a9621000 rw-p 00000000 00:00 0 
2b62a9621000-2b62ac000000 ---p 00000000 00:00 0 
2b62ac000000-2b62ac021000 rw-p 00000000 00:00 0 
2b62ac021000-2b62b0000000 ---p 00000000 00:00 0 
7f0000000000-7f0003000000 rw-p 00000000 00:00 0 
7f1000000000-7f1000200000 rw-p 00000000 00:00 0 
7f2000000000-7f2003400000 rw-p 00000000 00:00 0 
7fff4db40000-7fff4db66000 rw-p 00000000 00:00 0                          [stack]
7fff4dbba000-7fff4dbbc000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
forrtl: error (76): Abort trap signal
Image              PC                Routine            Line        Source             
libc-2.17.so       00002B6037BF3400  Unknown               Unknown  Unknown
libc-2.17.so       00002B6037BF3387  gsignal               Unknown  Unknown
libc-2.17.so       00002B6037BF4A78  abort                 Unknown  Unknown
libc-2.17.so       00002B6037C35ED7  Unknown               Unknown  Unknown
libc-2.17.so       00002B6037C3C3E4  Unknown               Unknown  Unknown
libc-2.17.so       00002B6037C3FBA8  Unknown               Unknown  Unknown
libc-2.17.so       00002B6037C426FC  __libc_malloc         Unknown  Unknown
rmapsnow-all.exe   0000000000A6272C  Unknown               Unknown  Unknown
rmapsnow-all.exe   0000000000A61292  Unknown               Unknown  Unknown
rmapsnow-all.exe   0000000000A5D771  Unknown               Unknown  Unknown
rmapsnow-all.exe   0000000000A5A0E7  Unknown               Unknown  Unknown
rmapsnow-all.exe   0000000000A22253  Unknown               Unknown  Unknown
rmapsnow-all.exe   0000000000411931  Unknown               Unknown  Unknown
rmapsnow-all.exe   00000000004118DD  Unknown               Unknown  Unknown
libc-2.17.so       00002B6037BDF555  __libc_start_main     Unknown  Unknown
rmapsnow-all.exe   00000000004117F6  Unknown               Unknown  Unknown
Any help is appreciated.
0 Kudos
1 Reply
Barbara_P_Intel
Moderator
541 Views

Moving this to the HPC Toolkit Forum. They know a LOT about MPI there.

0 Kudos
Reply