Problem: tbb scalable allocator does not return memory to OS

[bfurman]

Hi all :)

I investigate thread-safe tbb:scalable_allocator as a substitution for boost::pool. The pool is used for small structures (56 bytes on 32bit and 104 bytes on 64bit). Our product supports Linux/Windows 32bit/64bit and Apple OS X on 64bit. We want to support thread-safe life cycle for the structure as a step towards thread-safe product in the future. There are a lot of third party libraries linked to our product and most of them we do not build.

I compared performance and memory utilization metrics between tbb::scalable_allocator and boost::pool allocating 10,000,000 structures (560MB on 32bit and 1,040MB on 64bit). The performance, peak memory and allocation memory overhead metrics where satisfactory. But the deal-killer is free memory overhead. With tbb::scalable_allocator especially on 32bit platforms we cannot make memory freed with tbb::scalable_allocator available for regular new/malloc allocations.

Boost::pool provides a function to recover freed memory. It is slow, but we made it available to our product 'clear' manual commands. This boost::pool memory recover command returns to OS ~ 90% of freed memory. I did not find such a function in tbb::scalable_allocator.

It would not be a problem if we use tbb::scalable_allocator for _all_ our allocations, but we use native thread-safe malloc/new for our product and third party allocations. Just for the small structure we needed better performance/memory utilization and used boost::pool.

tbb 2.2 supports automatic replacement of OS allocator with its scalable memory allocator (Microsoft Windows* and Linux* OS). But it is not a solution for us, because first of all, MAC OS X is not supported. Secondly, the solution on Windows requires build-and/or-link that we cannot do to hundreds of third party libraries we use.

The only viable solution we see is to _use_ a tbb function that returns (recovers) freed memory to OS.

Do you know about memory recovering function in tbb:scalable_allocator?
Do you see another solution to our problem?

Thank you,
Boris Furman

[jimdempseyatthecove]

Boris,

During operational use of your application you will at some point reach a maximum number of in-use 56/105 objects. Afterwhich point the in-use number may go down and return back to near this maximum number. If you were to free this memory back into a general pool, then subsequently you may not be able to re-obtain this memory. What do you want to happen when you are unable to get one of these 56/105 byte objects?

Since your requirements are for a single object, as opposed to a general solution, why not write your own thread-safe allocator?

An alternative might be to use the TBB scalable allocator encapsulated within your own allocator object which keeps a count of these in-flight allocations. You would then add code to your application to take (in)action when the allocation count reaches thresholds.

Jim Dempsey

[Alexey-Kukanov · Accepted solution]

For now, there is no way to return all (or most) of memory back to OS. We are working on improvements in that area, one part of those being an implementation of memory pool. Please stay tuned.

Meanwhile, dynamic allocator replacement with the TBB implementation is not intrusive at all. On Windows, all you need to do is to add #include into a single (!) compilation unit for the main executable module or for a DLL that is loaded as part of process startup code. No need to add to hunderds of 3rd party libraries. But you are correct to say that Mac OS X is not supported.

[Vladimir_P_1234567890]

Quoting Alexey Kukanov (Intel)

But you are correct to say that Mac OS X is not supported.

But if you know how to add support for the library on mac os - welcome to contribute this!