Using task_arena to prevent deadlock due to task stealing

[richardl]

To start with a summary, I have had a similar deadlock to that described in https://software.intel.com/en-us/forums/topic/401006.  I can avoid the deadlock using task_arena, but that brings its own problem.  I am hoping for advice on these issues.

To quote e4lam’s very clear description of the problem “we have some tasks spawned which call function A(). A() locks a shared resource and then calls into other code that spawns more tasks. Due to the task stealing nature of TBB's scheduler, the worker thread that has A() on its stack may steal a task that runs A(). This causes a deadlock because A() has not finished yet, and can't because it is stuck earlier in the stack.”

It is not practical to avoid the locking involved in the deadlock.  This is, in part, because I work on only one component of a larger system.

I can prevent the deadlock by using a task_arena every time I call parallel_for, as sketched below.  (For simplicity, I use parallel_for to stand for any TBB function that runs tasks.)

      tbb::task_arena arena;
      arena.execute([](){tbb::parallel_for(...);});

But this brings two problems:

1) It degrades performance if only a small amount of work in done in each call to parallel_for.  In one, admittedly extreme, example involving many calls to parallel_for it caused an overall slowdown by a factor of more than three.  I can get round the slowdown by reusing the task_arena when there are multiple calls to parallel_for at the same level of nesting.  But this adds a complication that I would rather avoid.

2) I can’t reliably specify the total number of threads to be used by TBB.  Previously, I could create a task_scheduler_init and supplying it with the maximum number of threads.  But task_arena seems not to respect this limit. If there is only a single level of threading, I can get round this by specifying the number of threads when creating the task_arena.  But I have not found a way to specify an overall thread limit when there is nested threading involving task_arena.

As I say, advice on these issues would be welcome.

[richardl]

Hmm. I have received two email messages saying that Jeff W. (Intel) had commented on this thread, with the comment being "test" or "test 2".  But I don't see any replies when I look at the thread. Is there a problem?