More than 18 nodes of the MPI test run slowly

Intel® MPI Library

Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.

More than 18 nodes of the MPI test run slowly

405 Views

Hello,

Here is my environment information

Intel one API 2021.2

CentOS 7.6

MLNX_OFED_LINUX-5.3-1.0.0.1

UCX 1.10

I have two HCAs per node, but only one Active.

When I run a test of 18 nodes, I get results in 0.9 seconds.

[root@n00001 mpi-test]# time mpirun -np 18 -ppn 1 -f hostfile2 hostname
n07061
n07068
n07069
n07063
n07064
n07074
n07073
n07072
n07062
n07067
n07066
n07070
n07065
n07071
n07059
n07060
n07057
n07058

real 0m0.939s
user 0m0.841s
sys 0m0.871s

When I run tests with more than 18 nodes, I need 41 seconds to get results.

[root@n00001 mpi-test]# time mpirun -np 20 -ppn 1 -f hostfile2 hostname
n07066
n07070
n07073
n07067
n07075
n07068
n07065
n07069
n07074
n07071
n07072
n07076
n07059
n07057
n07058
n07060
n07064
n07063
n07061
n07062

real 0m41.026s
user 0m0.790s
sys 0m0.875s

Do you have any suggestions on this question? Thank you

Link Copied

0 Replies

Community support is provided Monday to Friday. Other contact methods are available here.

Intel does not verify all solutions, including but not limited to any file transfers that may appear in this community. Accordingly, Intel disclaims all express and implied warranties, including without limitation, the implied warranties of merchantability, fitness for a particular purpose, and non-infringement, as well as any warranty arising from course of performance, course of dealing, or usage in trade.

For more complete information about compiler optimizations, see our Optimization Notice.