Loading MKL slows down other libm function in centos 7.2, 7.3

shuoli — Fri, 20 May 2022 23:34:26 GMT

Hello,

After we load MKL libraries and execute a simple saxby in cblas libs, functions in libm e.g. lrint becomes 5-10X slower.

This seams to be in Centos 7.2 and 7.3 running on Haswell based CPU such as E5-2640 v4.

We have been using the attached file with gcc to test this issue

It seems MKL loader leaves upper 128 bits of the AVX registers in an unhealthy state and if we call _mm256_zeroupper(); right after loading MKL, the slowness goes away. We still don’t know why the problem only shows up on 7.2 and 7.3.

We can see that, lrint becomes ~6X slower after loading MKL.

-bash-4.2$ /nfs/DEV/PLT/kezhao/test/slowness_mkl_test

handle in 0.128975 sys 0.000000 usr 0.129086 double is 50000000000000.000000

-bash-4.2$ /nfs/DEV/PLT/kezhao/test/slowness_mkl_test /nfs/software/D4C/tflex_d4c_nightly/main/LSF/lib/MKL/4.0/

Succeed to load MKL lib /nfs/software/D4C/tflex_d4c_nightly/main/LSF/lib/MKL/4.0/

handle in 0.768661 sys 0.000000 usr 0.768906 double is 50000000000000.000000

Test environment:

-bash-4.2$ cat /etc/redhat-release

CentOS Linux release 7.3.1611 (Core)

-bash-4.2$ uname -a

Linux dn110706 3.10.0-514.el7.x86_64 #1 SMP Tue Nov 22 16:42:41 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

-bash-4.2$ lscpu|grep Intel

Vendor ID: GenuineIntel

Model name: Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz

Case	OS version	Glibc version	CPU type	MKL preloaded?	Slowness observed?
1	CentOS 7.2	2.17-105	Xeon E5-2640 v4	Yes	Yes
	CentOS 7.2	2.17-105	Xeon E5-2640 v4	No	No
2	CentOS 7.3	2.17-157	Xeon Gold 6130	Yes	No
	CentOS 7.3	2.17-157	Xeon Gold 6130	No	No
3	CentOS 7.3	2.17-157	Xeon E5-2640 v4	Yes	Yes
	CentOS 7.3	2.17-157	Xeon E5-2640 v4	No	No
4	CentOS 7.8	2.17-307	Xeon E5-2650 v4	Yes	No
	CentOS 7.8	2.17-307	Xeon E5-2650 v4	No	No
5	CentOS 6.7	2.12-1.166	Xeon Gold 6130	Yes	No
	CentOS 6.7	2.12-1.166	Xeon Gold 6130	No

Re: Loading MKL slows down other libm function in centos 7.2, 7.3

Khang_N_Intel — Thu, 26 May 2022 01:32:35 GMT

Hi Shuo,

I tried your code with the latest version of oneMKL, 2022.1, on Ubuntu 20.04 LTS and didn't see the slowdown that you mentioned.

Can you try the latest version of oneMKL and see if the problem goes away?

Best,

Khang

topic Loading MKL slows down other libm function in centos 7.2, 7.3 in Intel® oneAPI Math Kernel Library

Loading MKL slows down other libm function in centos 7.2, 7.3

Re: Loading MKL slows down other libm function in centos 7.2, 7.3