S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid>set I_MPI_DEBUG=4 S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid>set I_MPI_HYDRA_DEBUG=0 S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid>rem SOURCING IMPI 2021.10 RUNTIME S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid>call S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid\env\vars.bat Intel(R) processor family information utility, Version 2021.10 Build 20230619 Copyright (C) 2005-2023 Intel Corporation. All rights reserved. ===== Processor composition ===== Processor name : Intel(R) Xeon(R) Gold 6242R Packages(sockets) : 14 Cores : 24 Processors(CPUs) : 80 Cores per package : 1 Threads per core : 3 ===== Processor identification ===== Processor Thread Id. Core Id. Package Id. 0 0 0 0 1 1 0 0 2 0 1 0 3 1 1 0 4 0 2 0 5 1 2 0 6 0 0 2 7 1 0 2 8 0 1 2 9 1 1 2 10 0 2 2 11 1 2 2 12 0 0 4 13 1 0 4 14 0 1 4 15 1 1 4 16 0 2 4 17 1 2 4 18 0 0 6 19 1 0 6 20 0 1 6 21 1 1 6 22 0 2 6 23 1 2 6 24 0 0 8 25 1 0 8 26 0 1 8 27 1 1 8 28 0 2 8 29 1 2 8 30 0 0 10 31 1 0 10 32 0 1 10 33 1 1 10 34 0 2 10 35 0 0 12 36 1 0 12 37 0 1 12 38 1 1 12 39 0 2 12 40 0 0 1 41 1 0 1 42 0 1 1 43 1 1 1 44 0 2 1 45 1 2 1 46 0 0 3 47 1 0 3 48 0 1 3 49 1 1 3 50 0 2 3 51 1 2 3 52 0 0 5 53 1 0 5 54 0 1 5 55 1 1 5 56 0 2 5 57 1 2 5 58 0 0 7 59 1 0 7 60 0 1 7 61 1 1 7 62 0 2 7 63 1 2 7 64 0 0 9 65 1 0 9 66 0 1 9 67 1 1 9 68 0 2 9 69 1 2 9 70 0 0 11 71 1 0 11 72 0 1 11 73 1 1 11 74 0 2 11 75 0 0 13 76 1 0 13 77 0 1 13 78 1 1 13 79 0 2 13 ===== Placement on packages ===== Package Id. Core Id. Processors 0 0,1,2 (0,1)(2,3)(4,5) 2 0,1,2 (6,7)(8,9)(10,11) 4 0,1,2 (12,13)(14,15)(16,17) 6 0,1,2 (18,19)(20,21)(22,23) 8 0,1,2 (24,25)(26,27)(28,29) 10 0,1,2 (30,31)(32,33)34 12 0,1,2 (35,36)(37,38)39 1 0,1,2 (40,41)(42,43)(44,45) 3 0,1,2 (46,47)(48,49)(50,51) 5 0,1,2 (52,53)(54,55)(56,57) 7 0,1,2 (58,59)(60,61)(62,63) 9 0,1,2 (64,65)(66,67)(68,69) 11 0,1,2 (70,71)(72,73)74 13 0,1,2 (75,76)(77,78)79 ===== Cache sharing ===== Cache Size Processors L1 32 KB (0,1)(2,3)(4,5)(6,7)(8,9)(10,11)(12,13)(14,15)(16,17)(18,19)(20,21)(22,23)(24,25)(26,27)(28,29)(30,31)(32,33)(35,36)(37,38)(40,41)(42,43)(44,45)(46,47)(48,49)(50,51)(52,53)(54,55)(56,57)(58,59)(60,61)(62,63)(64,65)(66,67)(68,69)(70,71)(72,73)(75,76)(77,78) L2 1 MB (0,1)(2,3)(4,5)(6,7)(8,9)(10,11)(12,13)(14,15)(16,17)(18,19)(20,21)(22,23)(24,25)(26,27)(28,29)(30,31)(32,33)(35,36)(37,38)(40,41)(42,43)(44,45)(46,47)(48,49)(50,51)(52,53)(54,55)(56,57)(58,59)(60,61)(62,63)(64,65)(66,67)(68,69)(70,71)(72,73)(75,76)(77,78) L3 35 MB (0,1,2,3,4,5)(6,7,8,9,10,11)(12,13,14,15,16,17)(18,19,20,21,22,23)(24,25,26,27,28,29)(30,31,32,33,34)(35,36,37,38,39)(40,41,42,43,44,45)(46,47,48,49,50,51)(52,53,54,55,56,57)(58,59,60,61,62,63)(64,65,66,67,68,69)(70,71,72,73,74)(75,76,77,78,79) (eu-vw-num02:0,1,2,3,4,5,6,7,8,9,10,11,12,13,14) [0] MPI startup(): Intel(R) MPI Library, Version 2021.10 Build 20230619 [0] MPI startup(): Copyright (C) 2003-2023 Intel Corporation. All rights reserved. [0] MPI startup(): library kind: release [0] MPI startup(): libfabric version: 1.18.0-impi [0] MPI startup(): libfabric provider: tcp [0] MPI startup(): File "" not found [0] MPI startup(): Load tuning file: "S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid\env\..\etc\tuning_skx_shm-ofi.dat" [5] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [12] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [13] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [8] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [6] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [7] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [11] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [3] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [9] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [10] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [2] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [1] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [4] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [14] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Rank Pid Node name Pin cpu [0] MPI startup(): 0 66260 eu-vw-num02 0,1,2,3,4,64,65,69,72,76,78 [0] MPI startup(): 1 82024 eu-vw-num02 5,6,7,8,9,64,66,67,72,76,78 [0] MPI startup(): 2 65836 eu-vw-num02 10,11,12,13,14,65,69,72,76,78 [0] MPI startup(): 3 19816 eu-vw-num02 15,16,17,18,19,64,66,67,68,72,76,78 [0] MPI startup(): 4 16792 eu-vw-num02 20,21,22,23,24,64,65,66,67,68,69,70,71,72,76,78 [0] MPI startup(): 5 15076 eu-vw-num02 25,26,27,28,29,66,67,72,76,78 [0] MPI startup(): 6 76496 eu-vw-num02 30,31,32,33,34,64,69,72,76,78 [0] MPI startup(): 7 75804 eu-vw-num02 35,36,37,38,39,65,66,67,68,72,76,78 [0] MPI startup(): 8 70776 eu-vw-num02 0,1,2,3,4,64,67,68,72,76,78 [0] MPI startup(): 9 52692 eu-vw-num02 5,6,7,8,9,64,66,67,69,72,76,78 [0] MPI startup(): 10 75084 eu-vw-num02 10,11,12,13,14,64,65,66,72,76,78 [0] MPI startup(): 11 22580 eu-vw-num02 15,16,17,18,19,65,67,68,72,76,78 [0] MPI startup(): 12 60312 eu-vw-num02 20,21,22,23,24,64,65,68,72,76,78 [0] MPI startup(): 13 74484 eu-vw-num02 25,26,27,28,29,65,68,72,76,78 [0] MPI startup(): 14 74040 eu-vw-num02 30,31,32,33,34,64,67,69,72,76,78 #---------------------------------------------------------------- # Intel(R) MPI Benchmarks 2021.6, MPI-1 part #---------------------------------------------------------------- # Date : Fri Aug 25 00:31:54 2023 # Machine : Intel(R) 64 Family 6 Model 85 Stepping 7, GenuineIntel # Release : 6.2.9200 # Version : # MPI Version : 3.1 # MPI Thread Environment: # Calling sequence was: # IMB-MPI1.exe BiBand -npmin 15 -iter 1 # Minimum message length in bytes: 0 # Maximum message length in bytes: 4194304 # # MPI_Datatype : MPI_BYTE # MPI_Datatype for reductions : MPI_FLOAT # MPI_Op : MPI_SUM # # # List of Benchmarks to run: # Biband #--------------------------------------------------- # Benchmarking Biband # #processes = 14 #--------------------------------------------------- #bytes #repetitions Mbytes/sec Msg/sec 0 1 0.00 1888901 1 1 13.14 13137831 2 1 28.23 14115793 4 1 50.28 12571027 8 1 104.87 13108996 16 1 368.77 23048229 32 1 738.02 23063069 64 1 1695.31 26489275 128 1 3267.46 25527062 256 1 7502.08 29304988 512 1 5785.02 11298864 1024 1 11854.06 11576228 2048 1 22357.70 10916844 4096 1 30469.21 7438771 8192 1 22350.89 2728380 16384 1 17982.56 1097568 32768 1 25817.91 787900 65536 1 27732.25 423160 131072 1 27925.79 213057 262144 1 28196.41 107561 524288 1 30130.91 57470 1048576 1 31039.77 29602 2097152 1 32517.07 15505 4194304 1 28280.85 6743 # All processes entering MPI_Finalize (eu-vw-num02:0,1,2,3,4,5,6,7,8,9,10,11,12) [0] MPI startup(): Intel(R) MPI Library, Version 2021.10 Build 20230619 [0] MPI startup(): Copyright (C) 2003-2023 Intel Corporation. All rights reserved. [0] MPI startup(): library kind: release [0] MPI startup(): libfabric version: 1.18.0-impi [0] MPI startup(): libfabric provider: tcp [0] MPI startup(): File "" not found [0] MPI startup(): Load tuning file: "S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid\env\..\etc\tuning_skx_shm-ofi.dat" [5] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [2] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [12] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [4] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [7] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [8] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [9] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [11] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [10] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [3] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [6] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [1] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Rank Pid Node name Pin cpu [0] MPI startup(): 0 78096 eu-vw-num02 0,1,2,3,4,5,64,66,67,69,70,73,74,78,79 [0] MPI startup(): 1 50132 eu-vw-num02 6,7,8,9,10,11,64,66,68,69,71,73,74,78,79 [0] MPI startup(): 2 44600 eu-vw-num02 12,13,14,15,16,17,65,67,70,73,74,78,79 [0] MPI startup(): 3 77216 eu-vw-num02 18,19,20,21,22,23,66,67,68,69,70,73,74,78,79 [0] MPI startup(): 4 61268 eu-vw-num02 24,25,26,27,28,29,65,66,67,69,70,73,74,78,79 [0] MPI startup(): 5 81108 eu-vw-num02 30,31,32,33,34,35,65,66,67,68,70,73,74,78,79 [0] MPI startup(): 6 29248 eu-vw-num02 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,64,67,69,70,73,74,78,79 [0] MPI startup(): 7 6256 eu-vw-num02 2,3,4,5,6,7,64,68,70,73,74,78,79 [0] MPI startup(): 8 31184 eu-vw-num02 8,9,10,11,12,13,66,68,69,71,73,74,78,79 [0] MPI startup(): 9 60324 eu-vw-num02 14,15,16,17,18,19,64,65,67,70,73,74,78,79 [0] MPI startup(): 10 53076 eu-vw-num02 20,21,22,23,24,25,64,65,66,67,69,70,73,74,78,79 [0] MPI startup(): 11 82048 eu-vw-num02 26,27,28,29,30,31,65,66,68,70,73,74,78,79 [0] MPI startup(): 12 34760 eu-vw-num02 32,33,34,35,36,37,64,65,66,67,68,70,73,74,78,79 #---------------------------------------------------------------- # Intel(R) MPI Benchmarks 2021.6, MPI-1 part #---------------------------------------------------------------- # Date : Fri Aug 25 00:31:57 2023 # Machine : Intel(R) 64 Family 6 Model 85 Stepping 7, GenuineIntel # Release : 6.2.9200 # Version : # MPI Version : 3.1 # MPI Thread Environment: # Calling sequence was: # IMB-MPI1.exe BiBand -npmin 13 -iter 1 # Minimum message length in bytes: 0 # Maximum message length in bytes: 4194304 # # MPI_Datatype : MPI_BYTE # MPI_Datatype for reductions : MPI_FLOAT # MPI_Op : MPI_SUM # # # List of Benchmarks to run: # Biband #--------------------------------------------------- # Benchmarking Biband # #processes = 12 #--------------------------------------------------- #bytes #repetitions Mbytes/sec Msg/sec 0 1 0.00 1518462 1 1 11.80 11797235 2 1 24.17 12084972 4 1 88.85 22212590 8 1 186.61 23325738 16 1 369.29 23080397 32 1 678.90 21215470 64 1 1463.95 22874167 128 1 2709.97 21171606 256 1 5698.78 22260869 512 1 5657.78 11050359 1024 1 13547.49 13229973 2048 1 21944.39 10715034 4096 1 29655.69 7240160 8192 1 28906.30 3528601 16384 1 35382.53 2159578 32768 1 21692.34 661998 65536 1 24195.58 369195 131072 1 25225.74 192457 262144 1 24312.09 92743 524288 1 27516.60 52484 1048576 1 30010.56 28620 2097152 1 29999.25 14305 4194304 1 25224.62 6014 # All processes entering MPI_Finalize (eu-vw-num02:0,1,2,3,4,5,6,7,8,9,10,11,12,13) .\runtest.bat : [mpiexec@eu-vw-num02] check_downstream_work_complition (mpiexec.c:1303): downstream from host eu-vw-num02 exited abnormally At line:1 char:1 + .\runtest.bat *>> testout.txt + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : NotSpecified: ([mpiexec@eu-vw-...ited abnormally:String) [], RemoteException + FullyQualifiedErrorId : NativeCommandError [mpiexec@eu-vw-num02] check_downstream_work_complition (mpiexec.c:1307): trying to close other downstreams [mpiexec@eu-vw-num02] HYD_sock_write (..\windows\src\hydra_sock.c:387): write error (errno = 0) [mpiexec@eu-vw-num02] wmain (mpiexec.c:2275): assert (pg->intel.exitcodes != NULL) failed [mpiexec@eu-vw-num02] HYD_sock_write (..\windows\src\hydra_sock.c:387): write error (errno = 0) (eu-vw-num02:0,1,2,3,4,5,6,7,8,9,10,11,12,13) [0] MPI startup(): Intel(R) MPI Library, Version 2021.10 Build 20230619 [0] MPI startup(): Copyright (C) 2003-2023 Intel Corporation. All rights reserved. [0] MPI startup(): library kind: release [0] MPI startup(): libfabric version: 1.18.0-impi [0] MPI startup(): libfabric provider: tcp [0] MPI startup(): File "" not found [0] MPI startup(): Load tuning file: "S:\NUMECA_SOFTWARE\MPI\impi2021.10.0_gdavid\env\..\etc\tuning_skx_shm-ofi.dat" [11] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [10] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [4] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [1] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [9] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [7] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [13] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [8] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [2] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [3] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [6] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [5] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [12] MPI startup(): Imported environment partly inaccesible. Map=0 Info=0 [0] MPI startup(): Rank Pid Node name Pin cpu [0] MPI startup(): 0 74224 eu-vw-num02 0,1,2,3,4,72,73,75,77,78 [0] MPI startup(): 1 73984 eu-vw-num02 6,7,8,9,10,64,65,66,67,69,72,73,75,77,78 [0] MPI startup(): 2 78968 eu-vw-num02 12,13,14,15,16,65,66,67,69,72,73,75,77,78 [0] MPI startup(): 3 42468 eu-vw-num02 18,19,20,21,22,64,66,72,73,75,77,78 [0] MPI startup(): 4 46008 eu-vw-num02 24,25,26,27,28,69,72,73,75,77,78 [0] MPI startup(): 5 4788 eu-vw-num02 30,31,32,33,34,66,67,68,72,73,75,77,78 [0] MPI startup(): 6 25164 eu-vw-num02 35,36,37,38,39,65,67,69,72,73,75,77,78 [0] MPI startup(): 7 20008 eu-vw-num02 0,1,2,3,4,64,67,72,73,75,77,78 [0] MPI startup(): 8 21348 eu-vw-num02 6,7,8,9,10,64,67,69,72,73,75,77,78 [0] MPI startup(): 9 28200 eu-vw-num02 12,13,14,15,16,64,65,67,68,69,70,71,73,75,77,78 [0] MPI startup(): 10 82064 eu-vw-num02 18,19,20,21,22,64,66,68,72,73,75,77,78 [0] MPI startup(): 11 20340 eu-vw-num02 24,25,26,27,28,66,67,72,73,75,77,78 [0] MPI startup(): 12 80484 eu-vw-num02 30,31,32,33,34,64,67,68,72,73,75,77,78 [0] MPI startup(): 13 63812 eu-vw-num02 35,36,37,38,39,66,68,69,70,71,73,75,77,78 #---------------------------------------------------------------- # Intel(R) MPI Benchmarks 2021.6, MPI-1 part #---------------------------------------------------------------- # Date : Fri Aug 25 00:32:02 2023 # Machine : Intel(R) 64 Family 6 Model 85 Stepping 7, GenuineIntel # Release : 6.2.9200 # Version : # MPI Version : 3.1 # MPI Thread Environment: # Calling sequence was: # IMB-MPI1.exe BiBand -npmin 14 -iter 1 # Minimum message length in bytes: 0 # Maximum message length in bytes: 4194304 # # MPI_Datatype : MPI_BYTE # MPI_Datatype for reductions : MPI_FLOAT # MPI_Op : MPI_SUM # # # List of Benchmarks to run: # Biband #--------------------------------------------------- # Benchmarking Biband # #processes = 14 #--------------------------------------------------- #bytes #repetitions Mbytes/sec Msg/sec 0 1 0.00 2194868 1 1 13.97 13967266 2 1 42.29 21144542 4 1 102.40 25600007 8 1 206.57 25821324 16 1 329.75 20609541 32 1 800.89 25027936 64 1 1577.55 24649243 128 1 3163.81 24717242 256 1 6741.40 26333580 512 1 7151.24 13967266 1024 1 9916.28 9683870 2048 1 23052.86 11256280 4096 1 29260.64 7143711 8192 1 29052.17 3546408 16384 1 40318.77 2460862 32768 1 23749.83 724787 65536 1 27125.34 413900 131072 1 27721.44 211498 262144 1 28529.04 108830 524288 1 30175.82 57556 1048576 1 31527.50 30067 2097152 1 29696.76 14161 4194304 1 28131.08 6707 # All processes entering MPI_Finalize