topic Re:Mapping and pinning MPI tasks and threads in Intel® MPI Library

Mapping and pinning MPI tasks and OpenMP threads

cristmoltisanti — Thu, 18 Feb 2021 12:07:32 GMT

I am very confused with the Intel MPI and OpenMP mapping and I was hopping someone could help me to understand. I'll describe my problem with an example.

I want to run an application (LAMMPS) which uses MPI and OpenMP. The way that OpenMP is used here is via SMT and through the LAMMPS Intel package. The target architecture (compute node) is a 2x socket, 32 core x socket, 2 SMT x core AMD chips.

* Full node (64 MPI tasks, 2 OMP threads)

export OMP_NUM_THREAD=2 mpirun -np 64 -pk intel 0 omp 2 -sf intel ./lmp

I think the above is correct, but how can I achieve the following:

* Half node ( split into 2 sockets, 16 each)

export OMP_NUM_THREADS=2 mpirun -np 32 -pk intel 0 omp 2 -sf intel ...

* Half node (all in 1 socket)

export OMP_NUM_THREADS=2 mpirun -np 32 -pk intel 0 omp 2 -sf intel ./lmp

I have been playing with the following variables, but I have not yet been able to do it:

I_MPI_PIN_DOMAIN I_MPI_PIN_PROCESSOR_EXCLUDE_LIST I_MPI_PIN_PROCESSOR_LIST KMP_AFFINITY

Re: Mapping and pinning MPI tasks and OpenMP threads

PrasanthD_intel — Wed, 24 Feb 2021 12:28:16 GMT

Hi Micheal,

I am sorry for the late reply.

As you have mentioned that you wanted to distribute the MPI ranks based on sockets.

As per your two use cases :

I) Half node ( split into 2 sockets, 16 each)

In this case, as you want to distribute 16 ranks on each socket by launching a total of 32 ranks. There is no need to do pinning as this the default behavior.

2)Half node (all in 1 socket)

To launch all threads in a single socket try this command

I_MPI_PIN_DOMAIN=socket:compact I_MPI_DEBUG=10 OMP_NUM_THREADS=2 mpirun -n 32 -ppn 2 ./a.out

For more information on how to use I_MPI_PIN_DOMAIN, please refer (Interoperability with OpenMP* API (intel.com))

Let us know if you face problems.

Regards

Prasanth

Re:Mapping and pinning MPI tasks and threads

PrasanthD_intel — Wed, 03 Mar 2021 08:11:34 GMT

Hi Micheal,

We haven't heard back from you.

Let us if the given command satisfies your pinning requirements?

Regards

Prasanth

Re:Mapping and pinning MPI tasks and threads

PrasanthD_intel — Tue, 09 Mar 2021 07:26:46 GMT

Hi Micheal,

We are closing this thread assuming your issue has been resolved. We will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.

Regards

Prasanth