Unable To Run Tasks Concurrently

[Karim_N_]

Hello,

I'm working on a Windows application that requires interactivity during retrieval of data that may take several minutes to fully complete.  The user requests data and we immediately show partial data as soon as it is available (a few seconds).  During this time, the user may request additional or new data and we want to show partial data for the new request as soon as possible even though previous requests are not complete.

I'm having difficultly running multiple tasks concurrently. Clearly I haven't designed the workflow properly.

I have a class that spawns requests through a custom tbb::task called "SetupDataRetrievalTask" below:

//set up and register context
bool setGlobalProgress = true;
auto pContext = std::make_unique<TaskGroupContext>("Starting data retrieval...", nullptr, setGlobalProgress);
nonstd::observer_ptr<TaskGroupContext> context(pContext.get());
contextID = _taskMgr->Register(std::move(pContext));

//set up and spawn task
auto callback = std::bind(&DataManager::onDataAvailable, this, std::placeholders::_1);
SetupDataRetrievalTask* t = new(tbb::task::allocate_root(*context))
SetupDataRetrievalTask(handles, this, context, callback);
tbb::task::spawn(*t);

This seems to be working as anticipated.  Data updates to the view level are propagated through the context.

However, the data is only updated for the initial task created.  If I run through the code a second time, there will be no updates until the first task is completely finished.  I was under the impression that I could spawn as many of these tasks as I wanted and they would be run concurrently, using up CPU resources.  What I'm seeing is that my CPU is barely used and the tasks are being executed sequentially.  That means I'm waiting several minutes for the first request to completely finish before I'm seeing any data on the second request.

Do I need to do something to make the scheduler run these additional tasks concurrently?  I'm clearly missing something in how I've set this up.

Thanks.

[Karim_N_]

Perhaps the following is the issue?

I'm creating a new task group context for every request, meaning each has its own implicit task group. I need to share the same context for all requests, which results in each task being added to the same group, and hence they can run concurrently?. 

Please correct me if my understanding is faulty. If there is a better alternative (multiple task groups?), that would be good to know as well.

EDIT: My thoughts above were incorrect.  Making 'pContext' a member of the class and using it for every request still results in every task running sequentially.

[Alexei_K_Intel]

Hi Karim,

I do not see any issues in your code snippet. Do you use an explicit tbb::task_scheduler_init? Do you use other threads or threading runtimes (e.g. OpenMP) in your application? Could your share the details of the SetupDataRetrievalTask class, please?

There is no need to use your own context if you do not want to manipulate with priorities or exceptions.

In addition, I would recommend to use tbb::task_group (or tbb::task::enqueue if you do not have a synchronization point to call tbb::task_group::wait).

Regards,
Alex

[Karim_N_]

Hi Alex,

Thank you for the quick response. After some investigation, it turns out the bottleneck was not in the way we're using TBB, but a third-party library that does not support concurrent reads.  That being said, here is more information.

1) We are using one tbb::task_scheduler_init at the start of the application.

2) We are not using any other threading libraries.

3) SetupDataRetrievalTask:

class RetrieveDataTask : public TaskBase
{
public:
   RetrieveDataTask(const std::vector<TSHandle>& tsHandles,
      Manager* mgr, nonstd::observer_ptr<TaskGroupContext> context, DataCallback callback)
      : TaskBase(context), _tsHandles(tsHandles), _mgr(mgr), _callback(callback)
   {
   }

   ~RetrieveDataTask()
   {
   }

   tbb::task* execute()
   {
      std::vector<Request> requests;
      _mgr->initializeRequest(_tsHandles, requests, _context);
      _mgr->queueFetchData(requests, _callback, _context);
      return nullptr;
   }

private:
   std::vector<TSHandle> _tsHandles;
   Manager* _mgr;
   DataCallback _callback;
};


class SetupDataRetrievalTask : public TaskBase
{
public:
   SetupDataRetrievalTask(const std::vector<TSHandle>& tsHandles,
      DataManager* mgr, nonstd::observer_ptr<TaskGroupContext> context, DataCallback callback)
      : TaskBase(context), _tsHandles(tsHandles), _mgr(mgr), _callback(callback)
   {
   }

   ~SetupDataRetrievalTask()
   {
   }

   tbb::task* execute()
   {
      tbb::empty_task* continuation = new(this->allocate_continuation()) tbb::empty_task;
      continuation->increment_ref_count();
      RetrieveDataTask* rdt = new(continuation->allocate_child()) RetrieveDataTask(_tsHandles, _mgr, _context, _callback);
      tbb::task::spawn(*rdt);

      return nullptr;
   }

private:
   std::vector<TSHandle> _tsHandles;
   DataManager* _mgr;
   DataCallback _callback;
};

4) The context is crucial for me as I am using it extensively to monitor and report progress of running tasks. It seems to be a mechanism that is working well.  At the highest level (GUI), a context is created, setting up the necessary callbacks for progress reporting.  All tasks at lower layers share this context and can update as necessary.

5) I tried using tbb::task_group but it was difficult to convert the existing task to a functor. I was able to call "run" by pointing to the execute method of the task instance but the code became messy.  I haven't yet look at tbb::task::enqueue.  Is there a benefit to using it instead of spawning a task immediately?

Thanks.