topic On most Intel processors in Software Archive

MKL DGEMM Hyperthreading.

seyedalireza_y_ — Wed, 24 Aug 2016 16:57:53 GMT

Hi,

I'm trying to call DGEMM on relatiely big matrices (m=10000 , n=100000, k=10000) on knights landing.

When I’m profiling using vtune, I can see that a call to MKL DGEMM, is having 68 threads working (which is the number of physical cores) but the expectation is that it uses 272 threads (logical cores) because of hyper-threading. Other parts of my code where i use (openmp simd) directives, are using up to 272 threads. I'm wondering if there is any settings i need to setup in order to get hyper-threading working for my case.

Thanks,

Ali

Did you look into MKL_DYNAMIC

TimP — Thu, 25 Aug 2016 11:52:58 GMT

Did you look into MKL_DYNAMIC setting? Did you find an advantage in using all the logical threads (vs. spreading a smaller number evenly across cores) in your omp parallel (not simd alone) regions?

MKL_NUM_THREADS

Gregg_S_Intel — Tue, 06 Sep 2016 14:56:55 GMT

MKL_NUM_THREADS

Note that many applications perform best using fewer than 4 threads per core.

See this article for information about setting the number of threads per core, https://software.intel.com/en-us/articles/process-and-thread-affinity-for-intel-xeon-phi-processors-x200

On most Intel processors

McCalpinJohn — Tue, 06 Sep 2016 20:37:52 GMT

On most Intel processors DGEMM only uses one thread per core. The code is very tightly constructed (with very careful cache blocking) to give the best performance in this configuration. There are very few avoidable stalls that could be overlapped with work in the other logical processor. Using the other logical processor would cut the available cache in half, which would reduce the block sizes, increase the cache miss rates, and decrease the overall performance.

I have not looked at Intel's DGEMM implementation for Xeon Phi x200, but it is easy to believe that it has the same properties. (The first generation Xeon Phi (Knights Corner) was an exception because a single thread could only issue instructions every other cycle, so two threads were required to reach maximum speed on compute-bound codes. This limitation is not present in the second generation Xeon Phi (Knights Landing) -- one thread of execution can issue two instructions every cycle, getting reasonably close to peak performance.

Thank you everyone for your

seyedalireza_y_ — Tue, 25 Oct 2016 14:57:00 GMT

Thank you everyone for your thorough help and explanations, I tried and saw that actually as pointed by Dr. McCalpin, it's better not to override the setting, as the performance is declined when using more threads per core and the code already seems to reach peak performance.

1. I would also recommend to

SergeyKostrov — Fri, 09 Dec 2016 20:35:02 GMT

1. I would also recommend to look at compact and scatter settings for KMP_AFFINITY environment variable. 2. My extensive experience with xGEMM MKL functions shows that all these functions are very optimized when it comes to threading and it could be also controlled with OMP_NUM_THREADS and KMP_AFFINITY environment variables. 3. If, for example, a CPU with 4 hardware threads is used ( 4 cores, 8 logical CPUs ) then OMP_NUM_THREADS needs to be set to 4, and it doesn't improve performance if it is set to 8. 4. Take into account that a programmer's control is very simple and this is how it could look like: ... #ifdef _RTTHREADTOPU_BINDING_SHOWINFO _RTLIBAPI RTtchar g_szThreadToPU[] = RTU("KMP_AFFINITY=granularity=fine,proclist=[0,2,4,6],explicit,verbose"); #else _RTLIBAPI RTtchar g_szThreadToPU[] = RTU("KMP_AFFINITY=granularity=fine,proclist=[0,2,4,6],explicit"); #endif ...

>>...On most Intel processors

SergeyKostrov — Fri, 09 Dec 2016 20:36:58 GMT

>>...On most Intel processors DGEMM only uses one thread per core... Absolutely correct and I confirm that.

>>...I'm trying to call DGEMM

SergeyKostrov — Tue, 07 Feb 2017 00:19:14 GMT

>>...I'm trying to call DGEMM on relatiely big matrices (m=10000 , n=100000, k=10000) on knights landing... I finally completed a set of tests for 100Kx100K square dense matrices on a KNL server with 64 cores. Three are no any performance improvements if more than 64 threads are used. Here are tests results: ... Matrix multiplication C=A*B where matrix A( 114688x114688 ) and matrix B( 114688x114688 ) Allocating memory for matrices Intializing matrix data Matrix multiplication started Matrix multiplication completed at 1941.544 seconds Deallocating memory Processing Completed ...