Unexpected task_group behavior

[Lee__Mark]

I'm using elementary task_group functionality but am seeing unexpected behavior which is turning out to be problematic.

To summarize, here is a code snippet:

// This function is executed on the main application thread.
void Foo(tbb::task_group &tg)
{
    std::cout << "Main thread id is " << syscall(SYS_gettid);

    tg.run([&]() {
        std::cout << "Spawned worker, thread id is " << syscall(SYS_gettid);

        // Do a whole bunch of work using TBB thread pool.
    });

    // Do a whole bunch of work which has to be completed on the main thread.

    tg.wait();
}

From the TBB task_group help:

template<typename Func> void run( Func&& f )       Spawns a task to compute f() and returns immediately.

My expectation is that the main thread won't get involved with the work inside of the lambda and so can be used to execute, in parallel, others task which can only be processed by the main thread.

This is usually the case but I do observe the following happening on some executions:

Main thread id is 209946
Spawned worker, thread id is 209946

which is problematic since the main thread is now tied up with work which I'd prefer it not to be doing.

As I mentioned, this only happens seldom. Is this expected behavior (it seems to contradict the documentation), and if so would anyone have any suggestions for how to prevent this. Perhaps there are other idioms for robustly accomplishing what I need...

Thanks in advance,
Mark
 

[Lee__Mark]

Also I'm using TBB v2018.0.6 if that helps.

[Nikita_P_Intel]

Hi, Mark!

task_group::wait() method allows the main thread to participate in parallel work while waiting for the tasks in task_group to be completed - it is expected behavior.

[Lee__Mark]

Thanks Nikita.

Understood. It makes a lot of sense that the main thread would participate when task_group::wait() is called. What I'm asking about though is what conditions could cause it to participate when task_group::run() is called. One thought, for example, is what if there are no more threads in TBB's thread pool to allocate to the associated task arena when task_group::run() is called, then perhaps the main thread would be forced to execute the work itself. If that's the case, I am trying to figure out a solution for how I could avoid that from ever happening.

[Alexei_K_Intel]

TBB does not support such model because mandatory parallelism is not supported. I.e. you cannot expect that TBB task will ever be processed with some other thread beside a thread calling task_group::wait.

If you completely control everything in your application and fully understand all particularities when TBB worker threads will process the task group, you might consider a work around with an explicit task_arena. You can create the explicit task_arena and wait task group inside this task_arena:

tbb::task_group tg;
tg.run([]{});

tbb::task_arena a;
a.execute([&tg]{
    // CAUTION: deadlock or livelock is possible
    tg.wait();
});

A thread inside a task_arena can process only tasks inside that task_arena. However, the task was run in another task_arena (implicit to the calling thread), so the thread calling wait cannot process the task run outside the task_arena.

[Lee__Mark]

I understand now - the TBB decision for main thread to participate in the work is made when wait() is called. So in my example program, all of the main thread work (line 12) will have completed by that point. I mistakenly assumed from looking at my program output that task_group::run lambda was being executed there and then on the main thread, instead of what's actually going on, which is that the task is simply being added to a queue, which the main thread happens to pick up at a later point in time. Thanks both for your help!