Difference in behaviour when using a singlecustom MKL DLL vs "normal" linking

[AndrewC]

We have a heavily OMP multithreaded tool( Windows/64) that calls MKL 10.3(.11). To avoid "DLL hell" conflicts with other software that can call our code as a DLL ( e.g. matlab) we used the mkl/tools/builder makefiles to create a single/custom MKL DLL/.lib that we link against.

I have been trying to debug a 100% reproducible nasty crash on crtexit ( i.e. after exit(0) has been called). The stack stops with an access violation in the Windows API "HeapFree" called by the MKL DLL. I ran Inspector XE memory check to diagnose this, and could not find any issues.

Yet, I found that if OMP_NUM_THREADS=1, the problem goes away.

Now one would normally blame some nasty threading bug in my code for this, but I used Inspector XE thread check on "max" , but could not find any issue that would be trashing things.

As a last resort, I stopped using our"custom" MKL DLL, and linked against the regular MKL DLLs (mkl_core etc) and the problem disappeared.
Can anyone think why using a single custom DLL vs using the regular MKL DLL's would show different behaviour? It seems to be related to multi-threading in some way.

The build line for creating the custom DLL was....
nmake em64t buf_lib= - export=functions_list manifest=embed name=custom_mkl_em64t

[AndrewC]

Interestingly, switching back to the version of our custom DLL built from MKL 10.2.6 resolved the issue as well, so this issue seems to have come from changes in 10.3.

I have a wild guess ... on Windows DLL's can do things on DLL_ATTACH and DLL_DETACH when they are loaded and unloaded. Perhaps some of the MKL DLL's allocate/deallocate at these times. Of course, with a custom DLL all this behaviour is lost.

[AndrewC]

I have to retract the statement that switching back to 10.2.6 based custom DLL fixed the issue. It reduced the frequency of the issue, but did not eliminate it.
I have been running >1000 QA tests on a variety of input files and so far the only solution seems to be to switch from the one monolithic custom MKL DLL back to the "as-shipped" MKL DLLs.

[Andrey_G_Intel2]

Hello!

could you provide list of used MKL functions?

Andrey