[0:node44][../../dapl_module_poll.c:3972] Intel MPI fatal error: OpenIB-cma DTO operation posted for [2:node58] completed with error. status=0x1. cookie=0x40002
Assertion failed in file ../../dapl_module_poll.c at line 3973: 0
internal ABORT - process 0
[2:node58][../../dapl_module_poll.c:3972] Intel MPI fatal error: OpenIB-cma DTO operation posted for [0:node44] completed with error. status=0x6. cookie=0x40000
Assertion failed in file ../../dapl_module_poll.c at line 3973: 0
internal ABORT - process 2
[1:node44] unexpected disconnect completion event from [2:node58]
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
internal ABORT - process 1
[3:node58] unexpected disconnect completion event from [0:node44]
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
internal ABORT - process 3
[6:node28] unexpected disconnect completion event from [0:node44]
[7:node28] unexpected disconnect completion event from [0:node44]
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
internal ABORT - process 7
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
internal ABORT - process 6
[5:node59] unexpected disconnect completion event from [0:node44]
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
[4:node59] unexpected disconnect completion event from [0:node44]
Assertion failed in file ../../dapl_module_util.c at line 1593: 0
internal ABORT - process 4
internal ABORT - process 5
rank 0 in job 1 node44_33470 caused collective abort of all ranks
exit status of rank 0: return code 1
[Ending Job 36715]
Link Copied
For more complete information about compiler optimizations, see our Optimization Notice.