tbb concurrent_hash_map in shared memory

[yaron_y_]

I am using dpdk shared memory. So it "map the hugepage memory to the same virtual addresses in client and server applications"(exact memory mapping).

I need a hash between processes.

I can see from your documenatation that concurrent_hash_map  is good between threads.

If i use dpdk shared memory, and initialize concurrent_hash_map  in shared memory, will i be able to use concurrent_hash_map in shared memory same way as between threads?

[jimdempseyatthecove]

If your application(s) is setup as true inter-process shared memory as opposed to shadowed memory backed by some communication scheme, then at issue is the allocation of internal objects (and subsequent allocation of and/or expansion of allocatable sub-objects therein). While you can use placement new with a pointer into the inter-process shared memory to ctor the container (example: concurrent_hash_map), subsequent allocations by the ctor, or by some operation later, will have to have explicit allocators that redirects the allocation to the shared heap as opposed to local heap for the process (or TBB scalable allocator to local resources as the case may be).

Jim Dempsey

[yaron_y_]

Thanks for your answer

So i understand that it's possible, and that the problem will be with allocation after constructor.

Is there a shared memory allocator that i can use from tbb.

Or should i use boost allocator with managed_shared_memory::segment_manager?

jimdempseyatthecove wrote:

or by some operation later, will have to have explicit allocators that redirects the allocation to the shared heap as opposed to local heap for the process (or TBB scalable allocator to local resources as the case may be).

[RafSchietekat]

You should be able to pass the librte_malloc allocator to the shared concurrent_hash_map instance (which uses it to allocate its internal structures and contents), although you will have to provide the C++ glue yourself, it seems.

That Boost code might work too, but there's already a detail about NUMA that is not mentioned in Boost, and there might also be some overhead in Boost related to naming that you maybe don't need (I only had a quick look). I would start with this, and maybe add that C++ glue to librte_malloc later.

[Pashaei__Amir]

Hi, As you said I tried to allocate tbb concurrent hash map on shared memory using boost::shared_memory::allocator but it's not working because tbb does not use allocator::pointer (offset_ptr in boost case). Can you help me?