Memory leak with TBB 4.2 with high multi-threading

[Divya_S_1]

I'm not sure if this is a false positive or a real problem. I have tried and failed to find anyone else who's faced/fixed this yet.

Using TBB Version 4.2 (I can't upgrade due to legacy issues). I observe a memory leak within my application while using TBB concurrent containers in combination with a high load of TBB parallel threading.

A simple snippet to reproduce this issue  -

#include "tbb/tbb.h"

void testFunc()
{
	tbb::concurrent_unordered_map<int,int> t;
	tbb::parallel_for(1,100,[&](int p)
	{
		tbb::parallel_for(1,10000000,[&](int n)
		{
			t.insert(tbb::concurrent_unordered_map<int,int>::value_type(n,n));
		});
	});
	t.clear();
}
int main()
{
	testFunc();
	return 0;
}

This leaks about 500MB every time you execute testFunc. (Notice the memory usage before and after testFunc() is called).

I iterate again, you don't notice this when you are only running about 1000-100000 threads. 1 Million is where you can begin to notice chunks of memory being lost.

My question is -

1) Is this an old issue that's been fixed in the later TBB Versions?

2) Is it unnoticed because people don't generally tend to use concurrent containers that need to be so big/ are accessed/allocated along with such high threading loads?

3) Does anyone know a way to work around this?

My application deals with huge data such that I cannot do-away with multi-threading or know in advance the size of the containers necessary for my operations.

Any help would be appreciated! Thanks!

[Vladimir_P_1234567890]

hello,

Why do you think there a memory leak? I do not see that that you are trying to destroy the map.

--Vladimir

[Divya_S_1]

Oh, I thought I mentioned it, the code snippet is supposed to be inside any function. So when the container goes out of scope, I expect it to get destroyed and release memory. But I observe, that it doesn't. My bad, for not making it clear. I've updated the code now.

Also, an explicit call to clear() doesn't seem to have any impact either. 

I'm wondering if this could have any impact - my application uses the C++ memory allocators specifically and not tbb's own. I've tried switching them back as well, but I see the same problem. May be there's more than one config setting that I need to update?

Vladimir Polin (Intel) wrote:

hello,

Why do you think there a memory leak? I do not see that that you are trying to destroy the map.

--Vladimir

[Vladimir_P_1234567890]

It looks I also missed something. It starts to leak if you use 1 million tbb workers?

As I understand you have tried tbbmalloc as allocator also, right?

--Vladimir

[Divya_S_1]

Hi Vladimir, 

I just verified this by creating a brand new project and just adding the updated code snippet in the source file. No other libraries included. No other ops. I notice the same memory leak if I hit a break point after testFunc() is executed. This was using tbbmalloc.

100,000 workers leads to a leak in the order of 10s of MBs. Not very easy to notice.

Divya

[Vladimir_P_1234567890]

OK, I've changed the reproducer a bit.

#include "tbb/tbb.h"
#include "iostream"

void testFunc()
{
	tbb::concurrent_unordered_map<int,int> t;
	tbb::parallel_for(1,100,[&](int p)
	{
		tbb::parallel_for(1,10000000,[&](int n)
		{
			t.insert(tbb::concurrent_unordered_map<int,int>::value_type(n,n));
		});
	});
	t.clear();
}


int main()
{
	for (auto i=0; i<100; i++){
		tbb::task_scheduler_init init(100000);
		std::cout << "run " << i << std::endl;
		testFunc();
	}
	return 0;
}

This sample fails with out-of memory/out-of-resources error on Intel TBB 4.2 but with with Intel TBB 4.4 it runs till the end with 11.5 GB of stable average load from 'run' to 'run' in the loop. concurrent_unordered_map container was updated/refactored several times during last couple of years so you need to check the latest version.

--Vladimir

[Divya_S_1]

Hi Vladimir,

I'm sure it's not a out-of-memory issue for me as I have around 32GB RAM on my machine. Or is it not related to the system actually running out of memory? If it's the former, are you able to reproduce it with a smaller request that would fit your memory spec?

And I understand that there's probably a lot that's changed, but I also note the same issue with concurrent_unordered_set, a less heavy memory leak with concurrent_vector.

Like I said in my original post, I'm unable to move to a higher version due to legacy issues. I guess I'll just have to wait until we fix the dependencies and upgrade to the latest version of TBB :(

Divya

[RafSchietekat]

"tbb::task_scheduler_init init(100000);"

That's a joke, right?