tbbmalloc_proxy: memory not released

[diedler_f_]

Hi,

I use tbbmalloc_proxy.dll library in my application to enhance memory allocation. It works well but not all the memory is released.

For instance, I try to allocate 1 000 000 of Node* (my own class) stored in a conccurrent_hash_map then I delete them and clear the conccurrent_hash_map. I can see memory leaks on Windows for my process...

If I use regular allocation, I have no leaks.

Is there something to do in order to release all memory allocated by tbbmalloc_proxy ?

Thanks

[Maksim_D_Intel]

Hello,

I want just clarify that the tbbmalloc_proxy binaries can be used to replace memory allocation routines with the TBB scalable allocator across the whole process. 
Also, it looks like this question about memory leaks already described in http://stackoverflow.com/a/24146103/2230159 

If you consider another case, could you please provide a simple reproducer

[diedler_f_]

Maksim D. (Intel) wrote:

If you consider another case, could you please provide a simple reproducer

Hello,

Here a simple application that shows the problem, maybe it is a false positif...

unsigned long long nbIter = 1000000;
const size_t mbyte = 1024*1024;

// Windows function...
#include <windows.h>
#include <psapi.h>
static size_t getMemoryConsuption()
{
    PROCESS_MEMORY_COUNTERS pmc;
    GetProcessMemoryInfo(GetCurrentProcess(), &pmc, sizeof(pmc));
    return pmc.WorkingSetSize;
}

void test_memory()
{
    tbb::concurrent_unordered_set<int> _ht;

    _ht.rehash(100000000);

    std::cout << "memoryref = " << getMemoryConsuption()/mbyte <<  " Mb" << std::endl;
    for (unsigned long long i=0; i<nbIter; i++) {
        _ht.insert(i);
    }
    std::cout << "\tmemory = " << getMemoryConsuption()/mbyte <<  " Mb" << std::endl;
}

int main()
{
    for (unsigned int i=0; i<20; ++i)
    {
        test_memory();
    }
    return 0;
}

If I use tbbmalloc_proxy.dll, I have this output that show a memory leak :

memoryref = 3 Mb
        memory = 1198 Mb
memoryref = 80 Mb
        memory = 1198 Mb
memoryref = 338 Mb
        memory = 1198 Mb
memoryref = 335 Mb  <----- memory leak ?
        memory = 1198 Mb

...

If I use regular allocator :

memoryref = 3 Mb
        memory = 1197 Mb
memoryref = 4 Mb
        memory = 1198 Mb
memoryref = 4 Mb
        memory = 1198 Mb
memoryref = 5 Mb
        memory = 1198 Mb
...

There is no leaks in this case...

Why tbbmalloc_proxy not release memory ?

(just a question: what is the difference between tbbmalloc.dll and tbbmalloc_proxy.dll ?)

Thanks

[Alexei_K_Intel]

Hello,

The test_memory function is called many times and the consumption is stable (~335 MB for tbbmalloc and ~4 MB for the standard malloc). Therefore, we can suppose that it is not a memory leak but caching inside allocators. tbbmalloc uses complex caching techniques when the memory returned back to the allocator. It allows to speed up further allocations. You can try to use scalable_allocation_mode with the TBB_MALLOC_CLEAN_ALL_BUFFERS argument to force tbbmalloc clean up its internal caches. Usually, it is not very important how much memory is cached when all data is deallocated. The peak memory consumption is a more interesting characteristic because it is important not to use more memory than RAM available. In your case the both allocators use the same amount of memory (1198 MB) so I do not see a big issue here.

The tbbmalloc.dll library allows automatic replacement of allocator in your application without source change. You can just link it to your application and all malloc/free (new/delete) call will be redirected to tbbmalloc.dll. If you want to use tbbmalloc.dll directly, you need to call scalable_malloc/scalable_free.

Regards, Alex

[diedler_f_]

Thanks, it seems to work with TBB_MALLOC_CLEAN_ALL_BUFFERS

Best regards,