Loading MKL 2020 mkl_core.dll resets the DLL search order on Windows

[Christoph_Gohlke]

Hello,

this issue was reported at <https://github.com/numpy/numpy/issues/15431>

Loading the `mkl_core.dll` from MKL 2020 resets the DLL search order, probably by calling `SetDllDirectoryA(NULL)`.

This breaks any application relying on the DLL search order set by `SetDllDirectory`.

MKL 2019.5 does not reset the DLL search order.

Here's a Python 3.8 script using numpy+mkl and pywin32 demonstrating the issue:

Python 3.8.1 (tags/v3.8.1:1b293b6, Dec 18 2019, 23:11:46) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys, ctypes, win32api
>>> mklpath = sys.prefix + '/Lib/site-packages/numpy/DLLs'
>>> win32api.SetDllDirectory(mklpath)
>>> print(win32api.GetDllDirectory())
X:\Python38/Lib/site-packages/numpy/DLLs
>>> ctypes.CDLL(mklpath + '/mkl_core.dll')
<CDLL 'X:\Python38\Lib\site-packages\numpy\DLLs\mkl_core.dll', handle 7fff323a0000 at 0x11e427c0af0>
>>> print(win32api.GetDllDirectory())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
pywintypes.error: (0, 'GetDllDirectory', 'No error message is available')

[Gennady_F_Intel]

this is very strange as MKL themself knows nothing about the python. I would recommend addressing this question to the intel Pyhton Forum - https://software.intel.com/en-us/forums/intel-distribution-for-python

[Christoph_Gohlke]

This issue is not specific to Python. Python was just used to demonstrate the issue. Here's a C console application that demonstrates the same:

#include <iostream>
#include "windows.h"

int main()
{
    bool b = SetDllDirectoryA((LPCSTR)"C:\\Program Files (x86)\\IntelSWTools\\compilers_and_libraries_2020.0.166\\windows\\redist\\intel64_win\\mkl\\");
    std::cout << "SetDllDirectoryA " << (b ? "succeeded" : "failed") << "\n";

    char buffer[8192];
    DWORD ret = GetDllDirectoryA(8192, buffer);
    std::cout << "GetDllDirectoryA " << ((ret == 0 || GetLastError() != NO_ERROR) ? "failed" : "succeeded") << "\n";
    std::cout << buffer << "\n";

    HINSTANCE hDLL = LoadLibrary((LPCWSTR)L"mkl_core.dll");
    std::cout << "LoadLibrary " << ((hDLL == NULL) ? "failed" : "succeeded") << "\n";

    ret = GetDllDirectoryA(8192, buffer);
    std::cout << "GetDllDirectoryA " << ((ret == 0 || GetLastError() != NO_ERROR) ? "failed" : "succeeded") << "\n";
    std::cout << buffer << "\n";
}

Output:

SetDllDirectoryA succeeded
GetDllDirectoryA succeeded
C:\Program Files (x86)\IntelSWTools\compilers_and_libraries_2020.0.166\windows\redist\intel64_win\mkl\
LoadLibrary succeeded
GetDllDirectoryA failed

[Gennady_F_Intel]

Thanks for the update. that's really interesting and this is the first time we face such an issue. We will check. thanks again.

[Gennady_F_Intel]

the quick question  - which compiler option do you use to prevent this compile time error:   argument of type "LPCWSTR={const WCHAR={__wchar_t} *}" is incompatible with parameter of type "LPCSTR={const CHAR={char} *}"

[Christoph_Gohlke]

> which compiler option do you use to prevent this compile time error

Not sure. I just created a new C++ Console project using Visual Studio 2019.

Try to replace the offending line with:

HINSTANCE hDLL = LoadLibraryA((LPCSTR)"mkl_core.dll");

[Gennady_F_Intel]

ok, thanks.

I have 3 latest versions  ( 2018, 2019 and 2020 correspondingly) of MKL installed on my local systems and I see no problems on my part.

.... version 2018.....
SetDllDirectoryA succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2018\compilers_and_libraries_2018\windows\redist\intel64\mkl\
LoadLibrary succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2018\compilers_and_libraries_2018\windows\redist\intel64\mkl\

.... version 2019.....
SetDllDirectoryA succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2019\compilers_and_libraries_2019\windows\redist\intel64\mkl\
LoadLibrary succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2019\compilers_and_libraries_2019\windows\redist\intel64\mkl\

.... version 2020.....
SetDllDirectoryA succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2020\compilers_and_libraries_2020\windows\redist\intel64\mkl\
LoadLibrary succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2020\compilers_and_libraries_2020\windows\redist\intel64\mkl\

[Gennady_F_Intel]

one more thing from our side - the example of the code I used is attached.

compiling: cl test_mkl_loading.cpp 

cl version: Microsoft (R) C/C++ Optimizing Compiler Version 19.00.24215.1 for x64
Copyright (C) Microsoft Corporation.  All rights reserved.

[Christoph_Gohlke]

Hi,

I don't think you can load three different DLLs with the same name. What happens when you load the 2020 version first?

[Gennady_F_Intel]

yes, in the case if the only version is called the GetDllDirectoryA() reports the failure

SetDllDirectoryA succeeded
GetDllDirectoryA succeeded
C:\Apps\Intel2020\compilers_and_libraries_2020\windows\redist\intel64\mkl\
LoadLibrary succeeded
GetDllDirectoryA failed

[Gennady_F_Intel]

debugging the code under VS 2015 - I see the case is passed and the output buffer contains the correct directory...

[Gennady_F_Intel]

the issue is escalated and we will keep you informed when the fix of the problem will be available into official build of Intel MKL.

[Cross__Mat]

This issue has come up with some of our customers at NAG. Could you please let us know if there is a timescale for the fix? (Is it expected in 2020.1?)

Thanks.

[Gennady_F_Intel]

yes, the expected version is MKL 2020.1 and the ERT ~ the beginning of the next month. This thread would be updated. 

[Gennady_F_Intel]

The fix of the issue available in MKL 2020 update 1 which available for download.