Resource Leaks in TbbMalloc DLLs: A destructor TLSKey::~TLSKey is not called

[SergeyKostrov]

Memory Resource Leaks in TbbMalloc DLLs: A destructor TLSKey::~TLSKey is not called.

Verified with TBB v4.0 Updates 1 and 3. I'll provide more technical details some time later.

Best regards,
Sergey

[EDITED] I changed the title of the thread from 'Memory Leaks in TbbMalloc DLLs...' to 'Resource Leaks in TbbMalloc DLLs...'.

[SergeyKostrov]

Quoting Sergey Kostrov

Memory Resource Leaks in TbbMalloc DLLs: A destructor TLSKey::~TLSKey is not called.

Verified with TBB v4.0 Updates 1 and 3...

     ...
     Number of Memory Blocks: 1
     Size of Memory Block   : 32768 bytes
     Total Amount of Memory : 0.00 GB

     [ CRT malloc             ] All memory blocks are allocated -    0 ticks
     [ CRT free               ] All memory blocks are released  -    0 ticks
     Press ENTER to continue...

     DEBUG: scalable_allocator::allocate
     DEBUG: scalable_malloc
     DEBUG: internalMalloc
     DEBUG: isMallocInitialized
     DEBUG: doInitialization
     DEBUG: initMemoryManager
     DEBUG: ExtMemoryPool::init
     DEBUG: ExtMemoryPool::initTLS
     DEBUG: TLSKey::TLSKey
    DEBUG: TlsAlloc OK    - Index=20
    [ TBB scalable_allocator ] All memory blocks are allocated -    0 ticks
    [ TBB deallocate         ] All memory blocks are released  -    0 ticks
    Press ENTER to exit...
    ...
    Tests: Completed
    DEBUG: isMallocInitialized
    ...
    Completed
    ...

As you can see I used a 'printf' CRT function to display what methods / functions were called and
a destructor TLSKey::~TLSKey was not called. A breakpoint that was set in TLSKey::~TLSKey of the Visual Studio Debugger is also missed.

[Vladimir_P_1234567890]

Hello Sergey,

Allocator tls keys are not supposed to be destroyed until application is finished. They are destroyed upon exit by os. So behavoir is correct.

--Vladimir

[SergeyKostrov]

Quoting Vladimir Polin (Intel)

Allocator tls keys are not supposed to be destroyed until application is finished. They are destroyed upon exit by os. So behavoir is correct...

This is what MSDN says in some topic related to TLS management:

     ...
     If the threads of the process have allocated memory and stored a pointer to the memory in a TLS slot,
     they should free the memory before calling TlsFree. The TlsFree function does not free memory blocks
     whose addresses have been stored in the TLS slots associated with the TLS index. It is expected that
     DLLs call this function (if at all) only during DLL_PROCESS_DETACH.
     ...

As you can see there is an implicit recommendation to call TlsFree Win32 API function. Actually, I would name
the current TBB's solution as a Forced Resource Leaks.

I do almost the same in cases when significant amounts of memory allocated and application simply exits without
releasing the memory. Nobody wants to wait for 30 or 45 seconds, or even more, until all that memory will be properly released.

[Vladimir_P_1234567890]

Hello Sergey,

Sorry I did not get, your test case is shutting down for 30-45 secs more because of linking with tbbmalloc? We should definetely look into this issue. could you provide a test case please?

--Vladimir

[SergeyKostrov]

Quoting Vladimir Polin (Intel)

Sorry I did not get, your test case is shutting down for 30-45 secs more because of linking with tbbmalloc?

No. It happens when more than 1.5GB of memory is allocated with 'calloc' CRT function. I'll provide a screenshot
in order to give you some idea how slow that function is.

[SergeyKostrov]

Hi Vladimir,

Quoting Vladimir Polin (Intel)

Allocator tls keys are not supposed to be destroyed until application is finished. They are destroyed upon exit by os. So behavoir is correct...

This is what I found when I searched for a 'TlsFree' Win32 API function in TBB sources:
     ...
     ..\TBB40\Include\tbb\enumerable_thread_specific.h(241):            void destroy_key() { TlsFree(my_key); }
     ..\TBB40\Src\tbb\tls.h(66):    int  destroy()      { TlsFree(my_key); my_key=0; return 0; }
     ..\TBB40\Include\tbb\enumerable_thread_specific.h(241):            void destroy_key() { TlsFree(my_key); }
     ..\TBB40\Src\tbbmalloc\frontend.cpp(144):            TlsFree( Tid_key );
     ..\TBB40\Src\tbbmalloc\frontend.cpp(218):    TlsFree(TLS_pointer_key);
     Matching lines: 5    Matching files: 4    Total files searched: 192
     ...

As you can see 'TlsFree' function is used 4 times ( there is one duplicate in the search ) and there is a contradiction
with your statement '...They are destroyed upon exit by OS...'.

I also changed the title of the thread from 'Memory Leaks in TbbMalloc DLLs...' to 'Resource Leaks in TbbMalloc DLLs...'.

Best regards,
Sergey

[Vladimir_P_1234567890]

right, we are checking whether there are system resource leaks in case tls keys are not destroyed by the dll upon application exit() call since TlsAlloc() returns simple DWORD but not system handle.

Do you have any info that there is a real system resource leak?

thanks.

--Vladimir

[SergeyKostrov]

Hi Vladimir,

Quoting Vladimir Polin (Intel)

...Do you have any info that there is a real system resource leak?

No. I would do a very simple verification using Windows Task Manager:

     Task Manager -> View -> Select Columns -> Check 'Handle Count' & 'Thread Count'