topic The slowdown you experience in Intel® oneAPI Math Kernel Library

performance difference of DGEMM on matrix containing huge values

boreas — Fri, 16 Nov 2012 08:33:55 GMT

Dear MKL experts,

I've met an issue on matrix multiplication DGEMM routine that if the matrix cotnains huge values, i.e. 1.0d+17, the performance dropped significantly as low as 1/3 of regular scenario.

Any idea if this is possible and/or what the reason could be? I really don't expect such behavior. BTW, this is encountered only in Windows but not in Linux.

thanks,

Hi, thanks for the issue.

Gennady_F_Intel — Fri, 16 Nov 2012 11:23:10 GMT

Hi, thanks for the issue. Is that 32 or 64 bit systems?

boreas,

Gennady_F_Intel — Fri, 16 Nov 2012 11:31:24 GMT

boreas, Did you check the output result. Are there NaN into output results? and what type of CPU you are working? Gennady

Hello Gennady,

boreas — Fri, 16 Nov 2012 12:21:11 GMT

Hello Gennady, thanks for your reply. It is 64bit Windows Xp or Windows 7. no NaN in output results. we've reproduced on several system with intel processors intel core i7-2760QM intel Xeon X5550 thanks,

thanks,

Gennady_F_Intel — Fri, 16 Nov 2012 13:12:34 GMT

thanks, what is the problem size in that case? what is the version of MKL you use? ( I hope in the all cases threaded version of mkl have been used). /Gennady

It would also be interesting

Shane_S_Intel — Fri, 16 Nov 2012 16:22:39 GMT

It would also be interesting to know if there are any +/-Infs in your output.

hello Gennady and Shane,

boreas — Sat, 17 Nov 2012 03:20:28 GMT

hello Gennady and Shane, I use single thread, and as said, there is no NaN in output. there are lots of dgemm call with various sizes in my applicaiton, but a typical size is about M=3769, N = 32, K = 256

Sorry if I wasn't clear, but

Shane_S_Intel — Sat, 17 Nov 2012 05:29:29 GMT

Sorry if I wasn't clear, but I asked about INFs instead of NaNs ... they are distinct IEEE types. With such large inputs, it would seem natural to expect that overflows may be occuring and that the INFs are generated.

hello Shane,

boreas — Sat, 17 Nov 2012 06:07:41 GMT

hello Shane, from visual studio debugger, I do not see any INFs. also I am not sure what is the right way to check if there is any INFs. however, I do see lot of really tiny value, like 1.0d-69. not sure if this matters. And let me to illustrate what I was doing clearer it is matrix factorization { A, B; B, C}, where A contains some huge diagonal values, and C = C - B*INV(A)*B = C - B*INV(L) * INV(D) * INV(L) * B, if A = LDL. I use DGEMM to calculate the last outer product. Since INV(D) is involved, those tiny values appear. I monitored the slow down in DGEMM and thus I initiate this topic. Again, do you think those tiny value trigger anything like under flow or so? and does it matter to the performance? thank you so much.

The slowdown you experience

TimP — Sat, 17 Nov 2012 13:59:51 GMT

The slowdown you experience could be provoked by running in gradual underflow mode (the hardware default), when individual products produce results of magnitude < TINY(1d0). Assuming your main program is compiled by an Intel compiler, if you use an option such as /fp:source, you would follow by /Qftz to set abrupt underflow. Otherwise, you may require the C/C++ ftz intrinsic to set abrupt underflow. A Core I7-2 or -3 CPU should not exhibit the slowdown of earlier CPU models.

Hello TimP,

boreas — Sun, 18 Nov 2012 05:28:26 GMT

Hello TimP, thanks for your reply. But, by reading the INTEL fortran compiler document (my program is in Fortran), it looks like /Qftz is the default option since my program is compiled with /Os in release mode. So it should be already on. And I saw those tiny values in debugger because the debug module is compiled with /Qd which did not trigger /Qftz. Does this make sense? what should I try? -ftz or /Qftz Denormal results are flushed to zero. Every optimization option O level, except O0, sets -ftz and /Qftz.

Yes, an ifort main program

TimP — Sun, 18 Nov 2012 12:54:01 GMT

Yes, an ifort main program built with -O1 -O2 or -O3 would have /Qftz set, unless you set a /fp: option. In the latter case, you would follow up with /Qftz to get back to abrupt underflow. I guess ifort /Os is equivalent to /O1. This removes some major optimizations such as auto-vectorization, which would not be needed if all your time is spent in MKL. I was surprised by this, as -Os appears to be allowed in ifort only for Windows. I think the sentence with "leave flags as they are" in the documentation of /Qftz is misleading. /Qftz- does mean taking the hardware default, which is the 32-bit Windows default, but X64 Windows should set abrupt underflow before starting a .exe, so /Qftz- would generate code to set gradual underflow mode.

thanks, but this seems not

boreas — Sun, 18 Nov 2012 15:08:49 GMT

thanks, but this seems not explain why I got slow-down on x64 windows. I have /Qs, which should trigger /Qftz by default as there is no /fp used.

Can you give us the example

Gennady_F_Intel — Mon, 19 Nov 2012 05:29:38 GMT

Can you give us the example of this case to check the problem on our side?

unfortunately, a small

boreas — Tue, 20 Nov 2012 07:29:10 GMT

unfortunately, a small program can not reproduce this abnormal behavior. I'll get you once I could. thank you.

Hello Gennady, TimP, Shane,

boreas — Wed, 21 Nov 2012 08:20:55 GMT

Hello Gennady, TimP, Shane, I've attached a small example with the matrix data which contains tiny values. I built the code through VS2010 and Intel 12.1 compiler. All compiler options are default. here the behavior I observed is not exactly the same what I saw on the large code. For this example, all computations is supposed to be on MKL dgemm and therefore, I don't expect much performance difference between my debug and release exectuables, however, I do see that, 11.6 seconds w/ debug versus 0.35 seconds w/ release exectuable. the test is done with Intel I7-2760QM, 2.4GHz, w/ single thread Please help. the performance variation does confuse me. thank you.

Hello,

Gennady_F_Intel — Wed, 21 Nov 2012 10:20:20 GMT

Hello, Yes, I checked how it works on my side. The cause of the performance variation can be explained that intel compiler flushes denormals to zero by the default in release mode. in the debug mode /Qftz is off. on my local system ( win7, SNB, MKL 11.0.1, 64 bit) I had release - 0.45 sec debug - 12.4 sec debug with /Qftz - 0.51 sec

can you let me know where you

boreas — Wed, 21 Nov 2012 10:47:41 GMT

can you let me know where you put /Qftz? compiler option or linker option. details are appreciated.

one more question - if the

boreas — Wed, 21 Nov 2012 10:54:36 GMT

one more question - if the main program is with Intel C or Microsoft C, which option should I use? thank you.

The SSE intrinsics for

TimP — Wed, 21 Nov 2012 12:35:00 GMT

The SSE intrinsics for switching underflow mode are covered (with typos) here: http://software.intel.com/en-us/articles/how-to-avoid-performance-penalties-for-gradual-underflow-behavior Since the article was written, ifort adopted the Fortran standard method of setting underflow mode under USE ieee_arithmetic, so you can ignore the Fortran-bashing aspect of the article. call ieee_set_underflow_mode(.false.) Setting the initialization in main() by /Qftz/Qftz- is supported only by the Intel compilers. It takes effect at compile time, when you build main.obj. Qftz has no effect in the building of other .obj or at link time. By the way, the gcc equivalent of /Qftz is normally invoked by -ffast-math. With the SSE or language standard intrinsics, you can switch the mode at any point in your program (but don't do it inside a time-consuming loop). The usual practice of running in IEEE standard gradual underflow mode under MSVC and gcc probably motivated the changes in corei7-2 which are supposed to speed up the common cases.