[Bug report] Memory leak in CPU plugin

[Lundgaard__Thomas]

Hello,

I am experiencing a big memory leak with the attached model. It appears that ExecutableNetwork does not deallocate everything when it is destructed.

I have only tried the cpu plugin, so I do not know if it is present in other plugins as well.

I am running the latest version of openvino (2018.5.456) on a Windows 10 machine with a Xeon W2133 cpu.

Find attached the problematic model and a code snippet to reproduce.

When executing the code, the commit size observed in Windows Task Manager continues to increase by approximately 50 MB for every iteration.

Regards,

Thomas

[Lundgaard__Thomas]

Bump

[Shubha_R_Intel]

Hello Thomas. Here is the code in your main.cpp:

while (true)
{
    std::cout << "Press key to continue." << std::endl;
    std::cin.ignore();

    std::cout << "Loading inference plugin" << std::endl;
    auto plugin = PluginDispatcher({ L"" }).getPluginByDevice("CPU");

    CNNNetReader networkReader;
    networkReader.ReadNetwork("saved.xml");
    networkReader.ReadWeights("saved.bin");

    auto cnnNetwork = networkReader.getNetwork();
    cnnNetwork.setBatchSize(1);

    auto executableNetwork = plugin.LoadNetwork(cnnNetwork, {});
}

Your code has never left scope and all those objects are created via smart pointers. Why would you expect memory to be released while stuck in this while loop ? 

[Lundgaard__Thomas]

Hi,

The smart-pointers are local variables inside the loop and thus go out of scope at the end of each iteration of the loop. See for example here: https://stackoverflow.com/questions/6403055/object-destruction-in-c

[nikos1]

FWIW we are also seeing a statics order of destruction issue; not sure if this is related. Let's further debug this.

[Shubha_R_Intel]

Hello Thomas, (1) does not compile - came from your sample. However (2) does compile - comes from the OpenVino samples.

1) auto plugin = PluginDispatcher({ L"" }).getPluginByDevice("CPU")

2) InferencePlugin plugin = PluginDispatcher({ FLAGS_pp, "../../../lib/intel64" , "" }).getPluginByDevice(FLAGS_d);

Can you kindly clarify ?

Thanks,

Shubha

[Lundgaard__Thomas]

This is related to whether the UNICODE preprocessor symbol is defined. The classification sample compiles without UNICODE and in this case the line should be changed to read

auto plugin = PluginDispatcher({ "" }).getPluginByDevice("CPU");

Or just use the one from the classification sample. Both will provoke the memory leak.

PS: I used to compile with UNICODE set, but found a scenario where this gave rise to an access violation (segfault) when calling from some c# code. There is a post elsewhere on this forum mentioning that. I now compile without UNICODE and still experience the memory leak.

[Lundgaard__Thomas]

Were you able to reproduce this?

[Lundgaard__Thomas]

Hi again,

I have investigated this and have found the location of the leak. It is a thread-local work buffer in MKL-DNN that doesn't get deallocated because OpenVino calls the constructors and destructors of MKL-DNN primitives from different threads.

Specifically, the global_scratchpad_t object in https://github.com/opencv/dldt/blob/2018/inference-engine/thirdparty/mkl-dnn/src/common/scratchpad.cpp is allocated by a task created here https://github.com/opencv/dldt/blob/2018/inference-engine/src/mkldnn_plugin/mkldnn_graph.cpp#L1076 (and this allocation may therefore happen off the main thread). The scratchpad is deallocated when the InferenceEngine::ExecutableNetwork object in main() is destructed, which happens on the main thread. However, MKL-DNN uses thread local buffers and assumes that those two calls happen from the same thread.

The memory leak can be fixed by compiling MKL-DNN with preprocessor definition MKLDNN_ENABLE_CONCURRENT_EXEC. That may, however, increase memory usage (see https://github.com/intel/mkl-dnn/blob/master/cmake/options.cmake#L33). The real solution is probably to make sure that MKL-DNN primitives are constructed and destructed on the same thread.

- Thomas

[nikos1]

Good find Thomas, thank you! Hopefully this gets fixed soon.

I am going to try with MKLDNN_ENABLE_CONCURRENT_EXEC and keep an eye on memory usage to see if our issues get resolved.

cheers

nikos

[Shubha_R_Intel]

Hi Thomas, with your help I have reproduced this issue and filed a bug. Thank You !

Shubha

[Lundgaard__Thomas]

Hi Shubha,

Do you know if this is fixed in the 2019 R1 release?

[Shubha_R_Intel]

Dear Thomas, yes this leak has been fixed. Please download 2019 R1 and give it a test drive !

Thanks for using OpenVino !

Shubha

[Lundgaard__Thomas]

Hi Shubha,

I have just tried 2019 R1 and unfortunately I am still seeing this memory leak. Can you update the bug report with this information?

Thanks,

Thomas

[Shubha_R_Intel]

Dear Thomas, 

Wow. Sorry for this. I will certainly update the bug report accordingly.

Thanks,

Shubha

[Tsin__Ross]

As far as I know the google gflags library has memory leak problem

[Shubha_R_Intel]

Dear Thomas, this is not an OpenVino bug. Please review the following code:

//There is some leak per each LoadLibrary <-> FreeLibrary call on Windows. 

int main(int argc, char *argv[]) {

    LPCTSTR pluginName = "Full path to MKLDNNPlugin.dll";

    int c = 100;
    do { 

{         HMODULE object = LoadLibrary(pluginName); 

        FreeLibrary(object); 

         c–; 

    } 

 while (c > 0);
}

To avoid it, just to load plugin once:

InferencePlugin plugin = PluginDispatcher().getPluginByDevice("CPU");

[Tsin__Ross]

Hi Shubha, Could you answer my question here?

Shubha R. (Intel) wrote:

Dear Thomas, 

Wow. Sorry for this. I will certainly update the bug report accordingly.

Thanks,

Shubha

[Lundgaard__Thomas]

Hi Shubha,

Loading the plugin just once doesn't fix the leak either.

[nikos1]

Any update on this?

Is it an OpenVino bug or not?

If yes, ETA on a fix?

thanks!

Nikos

[Lundgaard__Thomas]

Hi.

Any updates on this?

Thanks, Thomas