Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.

Omp affinity thread binding issue?

Tue_B_
Novice
397 Views

Hi

I'm having some trouble with omp_places and seem to find contradictory information
From http://openmp.org/mp-documents/OpenMP_Examples_4.0.1.pdf

I find that:
omp_places={0:2}:8:2 
is equivalent to:
omp_places="{0,1},{2,3},{4,5},{6,7},{8,9},{10,11},{12,13},{14,15}"

In my code I automatically builds a string depending on the amount of cores detected in the system and user specification about numa spreading etc.

When there is no spreading I often end up with the following string:
OMP_PLACES={0:16}:1:16
which I would assume is equivalent to:
OMP_PLACES={0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

The problem is that the first one throws a warning about threads trying to get bound to ID 16-31 which is outside the range.

I'm guessing my issue is the last 16 in the "OMP_PLACES={0:16}:1:16" line, but I don't really understand why.

My understanding of what these parameters do are the following:
OMP_PLACES={a:b}:c:d

a = first thread ID in a group
b = number of threads to bind in this grouping
c = number of groups to create
d = the shift in first thread ID from one group to the following

Now obviously something is wrong with my understanding, so could anyone please enlighten me?

0 Kudos
4 Replies
TimP
Honored Contributor III
397 Views

With the Intel implementation of OpenMP, you may check the interpretation of OMP_PLACES by setting KMP_AFFINITY=verbose.

The verbose screen echo is strange in that it implies that OMP_PROC_BIND is invoked for affinity of some threads but not others.

I see that OMP_PLACES=cores is giving me the same on my platform as {0:2}:2:2 . This much appears to agree with what we read in the pdf at openmp.org . It seems in disagreement with the documentation of libiomp5 (in which OMP_PLACES is not supported for my platform).

If you care to read and test the open source OpenMP, if it appears to disagree with either the standard or with Intel libiomp5, you could ask on the Intel forum about open source omprtl.

0 Kudos
Tue_B_
Novice
397 Views

I will try the verbose setting tomorrow and see whether I can learn anything from that.

Regarding the {0;16} I am reasonably sure that does not mean thread 0 thru 16 but instead starting from 0 and then bind to the next 16 threads. Otherwise the example I have provided above from the OpenMp documentation would also not make any sense since they say:

omp_places={0:2}:8:2 
is equivalent to:
omp_places="{0,1},{2,3},{4,5},{6,7},{8,9},{10,11},{12,13},{14,15}"

but whereas if it was thread 0 thru 2 it would be something that at least starts of like this:

omp_places="{0,1,2},....

0 Kudos
jimdempseyatthecove
Honored Contributor III
397 Views

It means 0 thru 15 (16 total)

Jim Dempsey

0 Kudos
Tue_B_
Novice
397 Views

Having asked this question on both stack overflow and the openMp forum and from what I can read from the documentation, I believe this is a bug in the Intel implementation of omp_places:

Since:

OMP_PLACES={0:16}:1:16
should be equivalent to:
OMP_PLACES={0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15}

Yet one produces warnings and makes my program crash, when run through and outer wrapper program, while the other does not. My workaround was just to switch to the longer notation for the moment.

 

Link to other threads I have made on this topic:

http://openmp.org/forum/viewtopic.php?f=3&t=1933

https://stackoverflow.com/questions/37719216/equivalences-in-omp-places

 

0 Kudos
Reply