Showing results for

- Intel Community
- Software Development SDKs and Libraries
- Intel® oneAPI Threading Building Blocks & Intel® Threading Building Blocks
- concurrent_unordered_map: good hash functionsj for lists of 32-bit integers, character strings, etc

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

Highlighted
##

Hi,

I am disappointed with the lack of documentation regarding the hashing for concurrent_unordered_map. In particular, as a drop in replacement for boost's concurrent_unordered_map, they have completely different hash behavior.

In my application, I hash sorted vectors of 32-bit integers (std::size_t or unsigned int), With Boost I was using the 'PJW Hash Function'

From page 436 of the Aho Compiler Book (The one with the dragon on the cover)

Also seen here: http://www.cs.hmc.edu/~geoff/classes/hmc.cs070.200101/homework10/hashfuncs.html

The boost::unordered_map uses chained addressing and implements the canonical buckets with linked lists, and uses a prime number for the bucket sizes. You can see the large amount of explanation on it here: http://www.boost.org/doc/libs/1_49_0/doc/html/unordered/buckets.html

the concurrent_unordered_map uses a skip-order list. Essentially the data structure is a linked list of all your data, and the "buckets" are pointers into specific entries of this list. To be honest, even after reading this paper late last night, I'm still a bit confused as to the particular invariants and implementations, but, the biggest important difference, is that the paper seems to use a table size of size 2^k.

Consider (by example), taking any key (unsigned int) and reducing it mod 2^k. For example, if k = 1, then only the least significant bit of the key matters for determining a bucket. i.e. the lowest k significant bits determine the bucket. So a good hash here should enforce randomness on those k bits.

I'll be posting more to this soon with more explanation as I have it. I hope that those particularly well informed, could post more on this matter.

One question I have, is, if I know in advance the total size of this hash table, how should I choose the buckets as to minimize space and search time?

rhl_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

03-23-2012
06:54 AM

8 Views

concurrent_unordered_map: good hash functionsj for lists of 32-bit integers, character strings, etc

I am disappointed with the lack of documentation regarding the hashing for concurrent_unordered_map. In particular, as a drop in replacement for boost's concurrent_unordered_map, they have completely different hash behavior.

In my application, I hash sorted vectors of 32-bit integers (std::size_t or unsigned int), With Boost I was using the 'PJW Hash Function'

From page 436 of the Aho Compiler Book (The one with the dragon on the cover)

Also seen here: http://www.cs.hmc.edu/~geoff/classes/hmc.cs070.200101/homework10/hashfuncs.html

The boost::unordered_map uses chained addressing and implements the canonical buckets with linked lists, and uses a prime number for the bucket sizes. You can see the large amount of explanation on it here: http://www.boost.org/doc/libs/1_49_0/doc/html/unordered/buckets.html

the concurrent_unordered_map uses a skip-order list. Essentially the data structure is a linked list of all your data, and the "buckets" are pointers into specific entries of this list. To be honest, even after reading this paper late last night, I'm still a bit confused as to the particular invariants and implementations, but, the biggest important difference, is that the paper seems to use a table size of size 2^k.

Consider (by example), taking any key (unsigned int) and reducing it mod 2^k. For example, if k = 1, then only the least significant bit of the key matters for determining a bucket. i.e. the lowest k significant bits determine the bucket. So a good hash here should enforce randomness on those k bits.

I'll be posting more to this soon with more explanation as I have it. I hope that those particularly well informed, could post more on this matter.

One question I have, is, if I know in advance the total size of this hash table, how should I choose the buckets as to minimize space and search time?

1 Reply

Highlighted
##

With concurrent_hash_map you need good randomness in the low-order bits, so you could try if that also helps with concurrent_unordered_map.

RafSchietekat

Black Belt

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

03-23-2012
11:06 PM

8 Views

For more complete information about compiler optimizations, see our Optimization Notice.