Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.

tbb concurrent_hash_map in shared memory

yaron_y_
Beginner
662 Views

I am using dpdk shared memory. So it "map the hugepage memory to the same virtual addresses in client and server applications"(exact memory mapping).

I need a hash between processes.

I can see from your documenatation that concurrent_hash_map  is good between threads.

If i use dpdk shared memory, and initialize concurrent_hash_map  in shared memory, will i be able to use concurrent_hash_map in shared memory same way as between threads?

0 Kudos
4 Replies
jimdempseyatthecove
Honored Contributor III
662 Views

If your application(s) is setup as true inter-process shared memory as opposed to shadowed memory backed by some communication scheme, then at issue is the allocation of internal objects (and subsequent allocation of and/or expansion of allocatable sub-objects therein). While you can use placement new with a pointer into the inter-process shared memory to ctor the container (example: concurrent_hash_map), subsequent allocations by the ctor, or by some operation later, will have to have explicit allocators that redirects the allocation to the shared heap as opposed to local heap for the process (or TBB scalable allocator to local resources as the case may be).

Jim Dempsey

0 Kudos
yaron_y_
Beginner
662 Views

Thanks for your answer

So i understand that it's possible, and that the problem will be with allocation after constructor.

Is there a shared memory allocator that i can use from tbb.

Or should i use boost allocator with managed_shared_memory::segment_manager?

 

jimdempseyatthecove wrote:

or by some operation later, will have to have explicit allocators that redirects the allocation to the shared heap as opposed to local heap for the process (or TBB scalable allocator to local resources as the case may be).

 

 

0 Kudos
RafSchietekat
Valued Contributor III
662 Views

You should be able to pass the librte_malloc allocator to the shared concurrent_hash_map instance (which uses it to allocate its internal structures and contents), although you will have to provide the C++ glue yourself, it seems.

That Boost code might work too, but there's already a detail about NUMA that is not mentioned in Boost, and there might also be some overhead in Boost related to naming that you maybe don't need (I only had a quick look). I would start with this, and maybe add that C++ glue to librte_malloc later.

0 Kudos
Pashaei__Amir
Beginner
662 Views
Hi, As you said I tried to allocate tbb concurrent hash map on shared memory using boost::shared_memory::allocator but it's not working because tbb does not use allocator::pointer (offset_ptr in boost case). Can you help me?
0 Kudos
Reply