topic Re:Intel MPI pins processes but does not set the affinity mask on a Windows system in Intel® MPI Library

Intel MPI pins processes but does not set the affinity mask on a Windows system

ManOfKent — Wed, 22 May 2024 13:18:09 GMT

Intel MPI 2021.8 was used to run a 6 process local parallel job on a Windows system with a single 8 core CPU using the following environment variables:

I_MPI_DEBUG=4
I_MPI_PIN_PROCESSOR_LIST=0,1,2,3,4,5

Debug output shows:

[0] MPI startup(): Rank Pid Node name Pin cpu [0] MPI startup(): 0 33620 hostname 0 [0] MPI startup(): 1 27032 hostname 1 [0] MPI startup(): 2 25416 hostname 2 [0] MPI startup(): 3 29352 hostname 3 [0] MPI startup(): 4 6596 hostname 4 [0] MPI startup(): 5 9416 hostname 5

It is clear that the processes are being correctly pinned, as the load on the first 6 cores rises to 100% and the remaining 2 cores are virtually unloaded.

However, if the affinity mask is viewed in 'Process Explorer', or the affinity mask is obtained in the application using the Windows API function GetProcessAffinityMask(), then the affinity mask is 255 for all processes (i.e. apparently allowed to run on all 8 cores). This is unexpected.

If the same test is repeated with Microsoft MPI, with equivalent affinity settings (MPIEXEC_AFFINITY=2), then the affinity mask is 1,2,4,8,16,32 as expected.

Similarly if the test is repeated with Intel MPI on a Linux, the affinity mask is again as expected.

How does Intel MPI pin the processes on Windows and why does the affinity mask not reflect the process pinning? Is there an alternative way way to determine the affinity of each process programmatically from the application?

Many thanks in advance.

Re: Intel MPI pins processes but does not set the affinity mask on a Windows system

ManOfKent — Wed, 22 May 2024 15:40:28 GMT

After some more experimentation, I think I can answer my own question. I found that the affinity mask corresponding to the pinning applied by Intel can be obtained by using the Windows API function GetThreadGroupAffinity(). It looks as if Intel MPI sets the thread affinity rather than the process affinity.

Re:Intel MPI pins processes but does not set the affinity mask on a Windows system

Mahan — Mon, 10 Jun 2024 02:18:52 GMT

Hi Chris,

Intel MPI Library on Windows uses a different approach for process pinning than Linux. On Windows, Intel MPI Library does not set the process affinity mask directly. Instead, it uses the Windows Job Object feature to manage and control processes' CPU affinity.

The Job Object feature allows for more flexible and dynamic CPU affinity management, which is particularly useful in scenarios where processes may need to be migrated or rescheduled during their execution. By using Job Objects, Intel MPI Library can ensure that processes remain pinned to their assigned CPUs even if the system's CPU topology changes or if processes are rescheduled by the operating system.

This approach explains why the GetProcessAffinityMask() function returns a mask that appears to allow the process to run on all available CPUs, even though the processes are effectively pinned to specific CPUs by the Intel MPI Library's Job Object management.

Re:Intel MPI pins processes but does not set the affinity mask on a Windows system

Mahan — Wed, 29 May 2024 03:59:58 GMT

Hi Chris,

Please let me know if the last answer resolves the issue.

Re:Intel MPI pins processes but does not set the affinity mask on a Windows system

Mahan — Tue, 04 Jun 2024 04:37:50 GMT

Hi Chris,

Is there any update?

Re:Intel MPI pins processes but does not set the affinity mask on a Windows system

Mahan — Mon, 10 Jun 2024 02:17:28 GMT

I Chris,

I am closing this thread.

Please open a separate thread if you face any more issues.