Has performance on non-Intel processors been improved?

[AFog]

There is a long-standing controversy over whether software made with Intel compilers and libraries is deliberately crippling performance on non-Intel processors. See Wikipedia 

This also applies to the MKL, <removed>

However, the manual for the latest oneAPI compiler states about the /arch: option:

Code generated with these options should execute on any compatible, non-Intel processor with support for the corresponding instruction set.

See oneAPI manual 

This appears to be an important change of policy. Many programmers want to know if this change of policy also applies to function libraries, and in particular the MKL that has historically behaved poorly on non-Intel processors.

The MKL manual no longer contains the legal "Optimization notice" which previously stated that "Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors." Now it only includes the general catch-all disclaimer "Performance varies by use, configuration and other factors."

My preliminary testing seems to confirm that the performance of MKL on non-Intel processors has indeed been improved.

Can you confirm this? Many programmers have refrained from using Intel function libraries because they want good performance on all brands of CPUs. If the mechanism that dispatches by CPU vendor is no longer active then programmers may start to use MKL and other Intel libraries again. Therefore I would like to convey this information on my blog.

Agner Fog

[VidyalathaB_Intel]

Hi Agner,

Thanks for reaching out to us.

We are working on your issue. we will get back to you soon.

Regards,

Vidya.

[Gennady_F_Intel]

The Intel® oneAPI Math Kernel Library (oneMKL) is optimized for performance on Intel processors. oneMKL also runs on non-Intel x86-compatible processors. While many oneMKL optimizations are applicable to both non-Intel and Intel microarchitectures, performance on non-Intel microarchitectures can vary.

[AFog]

Thank you for the answer, Gennady.

Can you be more specific, please? Previous versions of MKL checked the CPUID vendor string and ran inferior versions of the code when the vendor string was not "GenuineIntel", as documented in my link.

This appears no longer to be the case.

The legal "Optimization notice" was required after legal settlements. I assume that these legal settlements are still in force. This means that the "Optimization notice" would still be legally required if the current version of oneMKL is checking the CPUID vendor string. I would therefore like you to answer this question: Does the removal of the "Optimization notice" reflect a change in the MKL code so that the functions no longer run different code versions depending on the CPUID vendor string? Does this apply to all functions in the library or only part of the library?

Your answer to this question is important so that I can tell my readers whether oneMKL can be recommended for software that is designed to run on all brands of x86 microprocessors.

[gil_moses]

Hi,

Unfortunately, in a benchmark session I conducted, there's still a performance hit when running on an AMD.

The comparison was done against an old version of the MKL libraries (pre-2020), and of course specifying the MKL_DEBUG_CPU_TYPE=5 environment variable.

The MKL under test was 2022.1.0, dated 3/2022 (installed using w_onemkl_p_2022.1.0.192_offline.exe).

The benchmarked products are audio plugins, and I opened 15 of them (that use MKL) in one session. The reported CPU was 85% using the newest MKL, and 50% with the older version + variable.

I made sure the new MKL didn't lose performance when updating a Windows Intel machine, meaning the performance degradation happened only on AMD.

I'm also interested to know where Intel stands regarding this point.

Thanks,

Gil.

[AFog]

Gil Moses, thanks for the report. What does the 85% mean? 85% of what?

Could it be because the AMD processor has AVX2 and the Intel processor has AVX512?

AMD processors with AVX512 are not available yet, AFAIK.

[gil_moses]

Hi Agner,

85% vs 50% were reported on the performance monitor of the app running the plugins, however it is not the actual CPU %. 

The numbers were given as a qualitative value, and I'm following with a more in-depth benchmark.

In both Intel and AMD cases, AVX(1) instructions were used.

If you happen to have a reliable benchmarking app I'll be glad to use it with the newest MKL.