We have some larger software project where many different algorithms use OpenMP and in some parts also MKL. To reduce CPU oversubscription we have set KMP_BLOCKTIME=0 which nicely solves the CPU oversubscription issue since many years.
Platform is Windows Server 2022 with MSVC++ 2019.
We are using these libraries
ippcore.dll 2018.0.3.1052
ippcv.dll 2018.0.3.1069
ippcvk0.dll 2018.0.3.1069
ippi.dll 2018.0.3.1083
ippik0.dll 2018.0.3.1083
ipps.dll 2018.0.3.1087
ippsk0.dll 2018.0.3.1087
libiomp5md.dll FileVersion: 5.0.2017.829,
libmmd.dll FileVersion: 18.0.0.0,
svml_dispmd.dll FileVersion: 18.0.0.0,
This works on all CPU platforms including I7 13700. But on a new Xeon Gold 6426Y when setting KMP_BLOCKTIME=0 we find a large amount of thread thrashing resulting in largely single threaded execution which results in much slower runtime (ca. 5 times slower). When setting KMP_BLOCKTIME=1 then the slowness goes away but the CPU oversubscription issue comes back. Are the libraries somewhat tuned to specific Intel CPUs and the 6426Y has a newer CPUID which causes the library to take a different code path?
Would updating the libraries help. I have noticed in the docs KMP_BLOCKTIME supports in later versions not only ms but also us and ns timings. Is that the solution I am looking for?
Below is a profiler screenshot where the threads are colored. The large light blue marked region should be running multithreaded like the other parts but with KMP_BLOCKTIME=0 on this specific CPU performance goes down. Is this a known issue?
The blocking stack is
| | | | | |- libiomp5md.dll!_vcomp_barrier |
| | | | | | |- libiomp5md.dll!__kmpc_barrier |
| | | | | | | |- libiomp5md.dll!__kmp_barrier |
| | | | | | | | |- libiomp5md.dll!__kmp_hyper_barrier_gather |
| | | | | | | | | |- libiomp5md.dll!__kmp_suspend_64 |
| | | | | | | | | | |- KernelBase.dll!WaitForSingleObjectEx |
while the releasing stack is
| | | | | |- libiomp5md.dll!_vcomp_barrier |
| | | | | | |- libiomp5md.dll!__kmpc_barrier |
| | | | | | | libiomp5md.dll!__kmp_barrier |
| | | | | | | |- libiomp5md.dll!__kmp_hyper_barrier_release |
| | | | | | | | |- libiomp5md.dll!__kmp_resume_64 |
| | | | | | | | |- KernelBase.dll!SetEvent |
Reducing the thread count to e.g. 4 did not lead to better performance because then the other regions are also affected by this.
Link Copied
1. w_onemkl_p_2023.2.0.49500
2.
#include <windows.h>
#include <stdio.h>
int main(int argc, char** argv)
{
const int N = 1000;
float* pValues = new float[N];
for (int k = 0; k < 100000; k++)
{
#pragma omp parallel
#pragma omp for schedule(dynamic)
for (int i = 1; i < N; i++)
{
pValues[i] = pValues[i - 1] * pValues[i];
}
}
}
3. Xeon6426Y
4. I was creating the chart with Windows Performance Toolkit.
Here is a chart of the test application with
- KMP_BLOCKTIME = 0,30,40,50
- With KMP_BLOCKTIME 50 then OMP_NUM_THREADS 12 and 32
The process duration is 126s with KMP_BLOCKTIME=0 vs 5s with KMP_BLOCKTIME 30. That is a huge difference.
This happens only on the Xeon6426Y CPU on my Intel I7 13700K and other similar CPUs with KMP_BLOCKTIME=0 this does not happen which is really strange.
I think I have found the issue. It is in __kmp_hyper_barrier_release which consumes on the Xeon Gold 6426Y much less CPU compared to all other platforms.
When dumping the instructions
we find at address 8005a375 f390 just one instruction before the biggest CPU diff
001`8005a375 f390 pauseand sure enough you did change the latency of the pause instruction again.
- Xeon Gold 6426Y Pause Latency: 9ns
- I7 13700 Pause Latency: 49 ns
- I5 6300U Pause Latency: 50 ns
- Xeon E5 1620 Pause Latency: 4 ns (10 year old CPU)
The newest Xeon has a latency of ca. 9ns which is about double as much as the old Haswell CPU of the Xeon E5 which has 4. But due to higher frequencies, better architectural changes the OpenMP code runs now so quick that the latency of the pause instruction matters and you do forget to spin even a little bit in an OpenMP loop with KMP_BLOCKTIME=0 which is the new? default anyway. That will ruin any decently sized OpenMP workload if these server grade CPUs are entering mainstream datacenters.
I am sure many people will find this and raise tickets about that one. Your support is going to have fun.
Where is this latency change documented? The Software-Optimization-Manual-V1-048 only talks about Skylake CPUs where the latency was raised to 140 cycles.
