Question on TBBMalloc

[Customer__Intel6]

Hi,
I am using TBBMalloc on Linux. Is there a way to preallocate the memory for TBB malloc?  I have a multithreaded C++ server application (threads are created on startup) that constructs and destroys mostly small objects (< 2Kbytes) but can occasionally based on input requests create bigger objects. It also uses TBB data structures extensively (concurrent_queue and concurrent_hash_map). The server preloads libtbmalloc_proxy.so.2 using LD_PRELOAD and forces libstdc++ to use it with GLIBCXX_FORCE_NEW environment variable. I am using TBB 4.4 Update5 on Linux 64bit (CentOS7). I investigated adding a MAP_POPULATE option to the mmap() call in MapMemory.h to avoid minor page faults and also tried to enable HugePages with the TBB_MALLOC_USE_HUGE_PAGES environment variable to reduce mmap() calls. But I still have some significant virtual memory allocation occuring in my server. I am trying to avoid mmap calls and page faults and would like them to be done at startup.

Thanks,
--Kannan.

[Vladimir_P_1234567890]

Hi Kannan,

Are you asking about memory pools? https://software.intel.com/en-us/blogs/2011/12/19/scalable-memory-pools-community-preview-feature

--Vladimir

[Customer__Intel6]

Hi Vladimir,

Thanks for the reply. No, I am not asking about the user memory pools. Using it would probably help but would require significant changes in my server (multiple layers, multiple libraries). I would like to pre size the default memory pool and allocate more blocks so that tbbmalloc would not ask the OS for memory.

--Kannan.

[Alexei_K_Intel]

Hi Kannan,

To work with preallocated memory you need a fixed memory pool. However, as you mentioned, it can require a lot of code refactoring. Perhaps, you can try to use a fixed memory pool in a critical part of your application.

Another option is try to limit the amount of memory that tbbmalloc requests from OS. To achieve it, you need to call the scalable_allocation_mode function with the TBBMALLOC_SET_SOFT_HEAP_LIMIT parameter. However, it does limit tbbmalloc completely because if there is no memory in internal caches, tbbmalloc will try to obtain it from OS. 

As for huge pages, did you use TBB_VERSION=1 to be sure that huge pages are enabled?

Regards, Alex

[Customer__Intel6]

Hi Alexei,

Thanks for the reply. I am trying to determine how much of a factor virtual memory operations (allocations and page faults) are having on my application performance. To do that I want to somehow pre configure TBB malloc library that is used extensively in my application to avoid these operations. From the responses, it looks like there is no easy way to do this. I will see if I can identify the impact in another way. 

For huge pages, I verified using linux /proc/meminfo that the allocated huge pages were used when I set the TBB_MALLOC_USE_HUGE_PAGES environment variable was set. I did not use TBB_VERSION=1.

--Kannan.