Comparing NUMA and MPI model

[octogone]

Hello!

Are MPI model and NUMA  [non-uniform memory architecture] mutually exclusive models? Can't MPI be used for a cluster with NUMA? What approach is the current trend?

Many thanks!

[TimP]

MPI is often run on multiple nodes, each node having NUMA architecture, but there's no necessary connection.  On a NUMA node, there is a possibility of aligning MPI ranks with NUMA partitions in hope of avoiding remote memory access, but it's not necessarily the best way.

[Gergana_S_Intel]

Just to add to Tim's response slightly: we would recommend running a couple of MPI ranks on a single NUMA machine but the majority of MPI communication will be happening across multiple nodes.  On a single NUMA machine, it might be better to do OpenMP or some other threading methodology as that has less memory overhead/requirements.

This is, of course, all dependent on your application and how it best uses the resources it has available.  Any MPI library should be able to run in a single NUMA machine out of the box.

Regards,
~Gergana