Intel MPI pins processes but does not set the affinity mask on a Windows system

[ManOfKent]

Intel MPI 2021.8 was used to run a 6 process local parallel job on a Windows system with a single 8 core CPU using the following environment variables:

I_MPI_DEBUG=4
I_MPI_PIN_PROCESSOR_LIST=0,1,2,3,4,5

Debug output shows:

[0] MPI startup(): Rank    Pid      Node name        Pin cpu
[0] MPI startup(): 0       33620    hostname         0
[0] MPI startup(): 1       27032    hostname         1
[0] MPI startup(): 2       25416    hostname         2
[0] MPI startup(): 3       29352    hostname         3
[0] MPI startup(): 4       6596     hostname         4
[0] MPI startup(): 5       9416     hostname         5

It is clear that the processes are being correctly pinned, as the load on the first 6 cores rises to 100% and the remaining 2 cores are virtually unloaded.

However, if the affinity mask is viewed in 'Process Explorer', or the affinity mask is obtained in the application using the Windows API function GetProcessAffinityMask(), then the affinity mask is 255 for all processes (i.e. apparently allowed to run on all 8 cores). This is unexpected.

If the same test is repeated with Microsoft MPI, with equivalent affinity settings (MPIEXEC_AFFINITY=2), then the affinity mask is 1,2,4,8,16,32 as expected.

Similarly if the test is repeated with Intel MPI on a Linux, the affinity mask is again as expected.

How does Intel MPI pin the processes on Windows and why does the affinity mask not reflect the process pinning? Is there an alternative way way to determine the affinity of each process programmatically from the application?

Many thanks in advance.

[ManOfKent]

After some more experimentation, I think I can answer my own question. I found that the affinity mask corresponding to the pinning applied by Intel can be obtained by using the Windows API function GetThreadGroupAffinity(). It looks as if Intel MPI sets the thread affinity rather than the process affinity.

[Mahan · Accepted solution]

Hi Chris,

Intel MPI Library on Windows uses a different approach for process pinning than Linux. On Windows, Intel MPI Library does not set the process affinity mask directly. Instead, it uses the Windows Job Object feature to manage and control processes' CPU affinity.

The Job Object feature allows for more flexible and dynamic CPU affinity management, which is particularly useful in scenarios where processes may need to be migrated or rescheduled during their execution. By using Job Objects, Intel MPI Library can ensure that processes remain pinned to their assigned CPUs even if the system's CPU topology changes or if processes are rescheduled by the operating system.

This approach explains why the GetProcessAffinityMask() function returns a mask that appears to allow the process to run on all available CPUs, even though the processes are effectively pinned to specific CPUs by the Intel MPI Library's Job Object management.

[Mahan]

Hi Chris,

Please let me know if the last answer resolves the issue.

[Mahan]

Hi Chris,

Is there any update?

[Mahan]

I Chris,

I am closing this thread.

Please open a separate thread if you face any more issues.