Memory Leak in MKL

[Gera__Danny]

Our program analyzes several images every few seconds. Each image is processed in its own thread. The threads are ordinary Windows threads. MKL functions are used in the process. We used a memory leak detection tool to find leaks in the application (Memory Validator by Software Verification). The tool revealed a leak in calls to MKL vsMul function (32 bits for each image). The call stack is as follows:

vsMul->mkl_vml_kernel_GetTTableIndex->_vmlGetThreadLocalData

I found a note in Intel documentation (https://software.intel.com/en-us/mkl-windows-developer-guide-avoiding-memory-leaks-in-intel-mkl) and accordingly I tried to use both the mkl_disable_fast_mm function and the MKL_DISABLE_FAST_MM environment variable. None of them helped to eliminate the memory leak.

How can I overcome this memory leak problem?

[Gennady_F_Intel]

Did you try to call mkl_free_buffers after the last call to Intel MKL functions?

[Gera__Danny]

Our application runs on an industrial machine and needs to be robust enough to run 24/7. Calling mkl_free_buffers before exiting the program does not help because the leaking buffers will continue to be accumulated as the program runs. Calling mkl_free-thread_buffers before exiting a thread may make more sense, but it is hard to do without compromising OOP structure (There are several types of algorithmic threads, only some of them call vsMul). I still don't understand why calling mkl_disable_fast_mm does nothing, or is there an inherent memory leak?

[Maria_K_Intel]

Hello,

That's correct. mkl_disable_fast_mm doesn't allow MKL to use fast internal memory allocations. What MKL version do you use? Could you please share a small reproducer with MKL functions that you use? It will help us a lot to figure out what causes a memory leak.

[Gera__Danny]

We use MKL version 2019.1.144. I tried to create a small example of the problem, but unfortunately the problem was not reproduced in a small example. I tried to call vsMul in a loop, each time in a different thread, and even several threads running concurrently, but the test showed no leak. I don't know what is different in our full application. Timing, maybe, or maybe the numbers in the vectors do matter.

[Gennady_F_Intel]

may be this is some kind of false positives reported by this tool. we never try such tool in the past. We usually use Intel Inspector to check same sort of problems ( deadlocks, mem leaks and etc...). 

[Gera__Danny]

I finally succeeded to create a small example with the leak problem. The trick is to perform the thread creation and calculation from within a DLL. I attach to the message my source files and the Memory validation tool report (as a PDF). The MKL_Test file is the program main. The MKL_TestAux files were compiled in a separate DLL that was called from the main program.

Any idea what is happening here?

[Gennady_F_Intel]

and did you try Intel(R) Inspector tool to check these leaks?

[Gera__Danny]

No, we don't have Inspector. 

So far, all the leaks our memory tool (C++ Memory Validator from Software Verification) found were real.

I would also add (maybe it is relevant) that we run 64 bit system on Visual Studio 2012.

[Gera__Danny]

Well, please, does anybody have any idea or had a look in the example I uploaded?

[Gera__Danny]

Well, it took me a while, but finally I got time to install Inspector and run the same test with Inspector analysis. The memory leak is still there, according to Inspector report. I attach here a zip file with all the analysis files. The source code was uploaded previously.

It seems like there is a real problem with our usage of MKL.

[Orra__Oriol]

Hello everyone,

I have the same issue as Danny.

I create some worker threads to do some calculations inside a DLL, and then does not deallocate the memory.

I am trying to use the functions: 

            MKL_Disable_Fast_MM();
             mkl_disable_fast_mm();

            MKL_Thread_Free_Buffers();
            mkl_thread_free_buffers();

            MKL_Free_Buffers();
            mkl_free_buffers();

But it is not making any difference, Could you give us a solution?

Thanks you very much,