Sender: LSF System Subject: Job 207702448: in cluster Exited Job was submitted from host by user in cluster at Wed Mar 9 09:22:39 2022 Job was executed on host(s) <36*eu-a6-013-13>, in queue , as user in cluster at Wed Mar 9 09:23:10 2022 <36*eu-a6-013-10> <36*eu-a6-013-04> <36*eu-a6-014-06> <36*eu-a6-014-12> <36*eu-a6-013-11> <36*eu-a6-014-08> <4*eu-a6-014-07> was used as the home directory. was used as the working directory. Started at Wed Mar 9 09:23:10 2022 Terminated at Wed Mar 9 09:23:29 2022 Results reported at Wed Mar 9 09:23:29 2022 Your job looked like: ------------------------------------------------------------ # LSBATCH: User input #BSUB -L /bin/bash #BSUB -W 00:05 #BSUB -n 256 #BSUB -e lsf_%J.err #BSUB -o lsf_%J.out #BSUB -J I22 # LOAD MODULES module purge module load intel/2022.1.2 #export I_MPI_OFI_PROVIDER=mlx #export I_MPI_FABRICS=shm:ofi #export UCX_TLS=ud,self,sm #export I_MPI_DEBUG=30 #export FI_LOG_LEVEL=debug export I_MPI_FABRICS=ofi # LAUNCH MPI mpirun ./mpi_hello_world ------------------------------------------------------------ Exited with exit code 143. Resource usage summary: CPU time : 149.00 sec. Max Memory : 4987 MB Average Memory : 16.00 MB Total Requested Memory : 262144.00 MB Delta Memory : 257157.00 MB Max Swap : - Max Processes : 8 Max Threads : 10 Run time : 18 sec. Turnaround time : 50 sec. The output (if any) follows: [1646814207.893422] [eu-a6-013-04:54662:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893418] [eu-a6-013-04:54663:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893416] [eu-a6-013-04:54667:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893414] [eu-a6-013-04:54669:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893415] [eu-a6-013-04:54670:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893484] [eu-a6-013-04:54653:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893444] [eu-a6-013-04:54654:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893509] [eu-a6-013-04:54655:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893449] [eu-a6-013-04:54656:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893441] [eu-a6-013-04:54661:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893436] [eu-a6-013-04:54666:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893505] [eu-a6-013-04:54668:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893577] [eu-a6-013-04:54657:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893551] [eu-a6-013-04:54658:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893551] [eu-a6-013-04:54659:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893582] [eu-a6-013-04:54660:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893577] [eu-a6-013-04:54664:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, [1646814207.893609] [eu-a6-013-04:54665:0] select.c:434 UCX ERROR no active messages transport to : posix/memory - Destination is unreachable, sysv/memory - Destination is unreachable, self/memory - Destination is unreachable, sockcm/sockaddr - no am bcopy, rc_verbs/mlx5_0:1 - Destination is unreachable, rc_mlx5/mlx5_0:1 - Destination is unreachable, PS: Read file for stderr output of this job.