Parallel for, lambda, and hashmaps

[Sensei_S_]

Dear all, 

I am suspicious I shouldn't to this, so I am asking for help and advice. I have a concurrent_hash_map, and I'd like to remove lots of values. For this, I'd like to write a parallel for with a lambda. It seems, however, that iterators are not supported:

    tbb::parallel_for(hash_.begin(), hash_.end(),
                      [](auto &p)
                      {
                          
                      });

The code above gives an error on the iterators, as the call expects integers. Is it possible to have a parallel_for with lambdas and iterators?

And one more question, if I may. I will be calling erase(const Key &k) in the lambda. Will it be safe to erase concurrently? I am worried all iterators will be invalid and my for gets lost the dark...

Thanks!

[Vladimir_P_1234567890]

did you try to use the blocked_range? it accepts iterators.

--Vladimir

[Vladimir_P_1234567890]

Iteration limitations

https://www.threadingbuildingblocks.org/docs/help/reference/containers_overview/concurrent_hash_map_cls/iterators_hash_map_cls.htm

Do not call concurrent operations

So erase() can't be called inside lambda in your case

--Vladimir

[Sensei_S_]

Thanks Vladimir, I'm not surprised but I wished I could! ;)

[jimdempseyatthecove]

Can you create a new hash map with the reduced set of entries, then either a) start a task to perform a sequential delete of the now unused hash map, or b) use blocked_range to delete all entries in the old container in parallel (this may require a cast).

Jim Dempsey

[Sensei_S_]

Thanks Jim, I am not sure if I understand b).

The problem I am facing is quite big, and cloning isn't really good, I will saturate RAM immediately. The b) option with blocked_range can be done in parallel, but I understand I could not erase things concurrently. 

Can you explain the b) option?

Thank you!

[RafSchietekat]

I don't think that b) is very realistic, because you can't iterate through a concurrent_hash_map while another thread might be erasing elements from it. Maybe this is a good use case for providing thread-safe access to different partitions, but that's not available now.

Probably the best you could do with the current API is to determine, in parallel, the elements you want to erase, and to record just their keys in a temporary container. When this is complete you can go through the temporary container, in parallel, and erase all those elements from the map based on the recorded keys.

(If the keys are also rather large, you just might be able to record accessors instead, but I couldn't say for sure without taking a closer look.)