concurrrent_unordered_map crash when insert() or find()

[Benyu_Z_]

I have a multithread program which uses concurrent_unordered_map (latest tbb version, built from src). It will crashes after several hours running. The core dump is

(gdb) where
#0 flist_iterator (other=..., this=<synthetic pointer>) at /usr/include/tbb/internal/_concurrent_unordered_impl.h:90
#1 get_bucket (bucket=<optimized out>, this=<optimized out>) at /usr/include/tbb/internal/_concurrent_unordered_impl.h:1333
#2 internal_insert (value=..., this=0x7f75a404e500) at /usr/include/tbb/internal/_concurrent_unordered_impl.h:1127
#3 insert (value=..., this=0x7f75a404e500) at /usr/include/tbb/internal/_concurrent_unordered_impl.h:860
#4 ngram_segment<20>::segment (this=this@entry=0x7f75a404e4f0, sentence=..., result=..., debug=debug@entry=false) at /usr/local/include/segment/segment.h:319

The code around 319 is:

{

auto it = dp_cache_.find(sentence);

if (it == dp_cache_.end()) {

  dp_cache_.insert(make_pair(sentence, result));

}

}

And the types:

typedef pair<float, vector<int>> segmentation;

concurrent_unordered_map<string, segmentation> dp_cache_;

I tried concurrent_hash_map and different versions of tbb or pre-built binaries. They all crashes. 

Anyone seeing similar cases? 

Helps are appreciated!

[Christophe_H_Intel]

Hi, Benyu,

I have a small program that seems to work, which I am attaching to this note.  The command line is

cumap_demo [ <insertions> [ <value-mask> [ <do-serially> [ <find-first> ] ] ] ]

where

<insertions> - number of total insertions to attempt

<value-mask> - maximum number of distinct values to insert (see below)

<do-serially> - 1 if you want it to execute serially

<find-first> - 1 if you want a find() before an insert()

random values in the range [1 : value-mask] are used as keys (after being converted to strings) in the concurrent_unordered_map, and the vector in the record keeps track of the indices that had this value.  I added a spin_mutex to the record to serialize operations on the vector.

I don't know if this resembles what you are trying to do.

Regards,
Chris

[Christophe_H_Intel]

Hi, Benyu,

Just noticed I left some older comments on compiling the demo in the file.  You can ignore the comments on -D values; I made them into command-line values so you don't have to recompile.

Regards,
Chris

[Benyu_Z_]

HI Chris,

thanks for the demo code. It looks good, but in my case, different threads may run insert_body() or check_body() at the same time. Would it cause some issues?

[Benyu_Z_]

Hi Chris,

It seems it's clear() which causes the crash. It runs well for one day after adding a guard around clear(). I will keep it running for one more day and see.

[RafSchietekat]

Just to make sure, what kind of guard are you using, and how? It doesn't help if the concurrent operations aren't also covered, and it defeats the purpose if the guard is used around individual supposedly concurrent operations...

[Benyu_Z_]

I used a RWSpinLock to guard the single thread which calls clear() (WriteHolder), and the multiple threads which call concurrent safe operations(ReadHolder). 

[RafSchietekat]

OK, although a kernel-level mutex might be more appropriate in this case; I'll assume that you implied that you do many concurrent operations between acquiring and releasing a read lock.

[Benyu_Z_]

Yes, your assumption is right. Any suggestion on the kernel-level mutex libraray/wrapper?

[Benyu_Z_]

BTW, Raf, I'm using https://github.com/facebook/folly/blob/master/folly/RWSpinLock.h.