Problems with Intel MPI

Palina_L_ · ‎05-13-2015

I have trouble with running Intel MPI on cluster with different different numbers of processors on nodes (12 and 32).

I use Intel MPI 4.0.3 and it works correctly on 20 nodes with 12 processors (Intel(Xeon(R)CPU X5650 @2.67)) at each, and all processors works correctly, then I try to run Intel MPI on other 3 nodes with 32 processors (Intel(Xeon(R)CPU E5-4620 v2@2.00) at each and they work correctly too.

But when I try to run my tasks on all nodes with different types of processors and the same type of Intel MPI I cant use more than 48 processors. Spead falls. I use option --machinefile

mpirun -machinefile mpihosts.txt ./wrf.exe

mpihosts.txt (cn01:12

cn02:12

...

cn29:32

cn30:32)

How can I use Intel MPI 4.0.3 correctly on all of these nodes?

Artem_R_Intel1 · ‎05-14-2015

Hi Palina,

Could you please clarify whether your application didn't work at all on such heterogeneous configuration (does it fail)? If so could you please provide corresponding error message (if any).
Is it possible for you to try newer Intel MPI Library versions?

Palina_L_ · ‎05-14-2015

Thank you for the answer!

The problem is that I do not get error when I try to run my application, it is better to say that it works more and more slowly, and if I try to use more than 48 processors on all nodes (on servers with 12 and 32 processors) my programm works very slower, then I use 48 and less processors. Although the total number of processors used on cluster more than 300.

Yes, I'm going to install a new version of the Intel MPI in 2015, may be it is solve the problem, but I would like to now the reasons?

Artem_R_Intel1 · ‎05-28-2015

Hi Palina,
As far as I know such heterogeneous (with different processor types) MPI runs weren't supported in the Intel MPI Library 4.0.3.
This functionality (if not mistaken) has been implemented in IMPI 4.1 version. I'd recommend you to try recent IMPI 5.x versions.
Additionally please see the description of I_MPI_PLATFORM environment variable in the Reference Manual - it may be useful for such runs with mixed processor types.