- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I have an exe that creates 5 tasks (let's call these tasks A tasks), push them onto a task_list and run them using spawn_root_and_wait(taskList). Before I call this I call task_scheduler_init init(25).
The above tasks create more tasks (let's call these tasks B tasks) and they are pushed on to the task scheduler using tbb::task::enqueue().
Here's when things get tricky, after A tasks push B tasks onto the task scheduler, they wait for the B's to finish. I implemented a wait using condition variables. Basically A's wait until B's come back and notify the condition variable.
What happens in my program is my code runs all the way until A's are waiting. But B's never get executed. I put a print statement in B's execute() method and that never gets executed. So it's like 1. TBB never got the tasks or 2. TBB is waiting for some threads to finish.
I want to say the reason is 2. Why? 1. I ran my code through the Intel Inspector, and it didn't spit out any warnings or errors. It just happily waits forever. 2. In the Windows Task Manager it shows only 5 threads for the exe eventhough I used task_scheduler_init with 25. So I'm guessing those 5 threads are just waiting running the A's and TBB doesn't have anymore threads to execute the B's.
Is this a possible case? In Windows Task Manager as the number of threads does it show all the TBB worker threads as well?
Thank you in advance.
Link Copied
13 Replies
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could this mean that lazy thread creation only occurs when waiting for children?
Anyway, "pushed on to the task scheduler" is highly ambiguous when there's a choice between spawn and enqueue, and you shouldn't wait outside the scheduler's knowledge (those TBB threads are just sitting there doing nothing).
Together this would explain the symptoms you describe.
Anyway, "pushed on to the task scheduler" is highly ambiguous when there's a choice between spawn and enqueue, and you shouldn't wait outside the scheduler's knowledge (those TBB threads are just sitting there doing nothing).
Together this would explain the symptoms you describe.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting dulantha_f
...2. In the Windows Task Manager it shows only 5 threads for the exe eventhough I used task_scheduler_init with 25. So I'm guessing
those 5 threads are just waiting running the A's and TBB doesn't have anymore threads to execute the B's...
[SergeyK] That looks very strange and I didn't experience something like this in my codes and test-cases.
If I called a 'task_scheduler_int( 255 )' a maxnumber of threads created would be 255.
Is this a possible case? In Windows Task Manager as the number of threads does it show all the TBB worker threads as well?
[SergeyK] If a thread is created it must be shown in the Task Manager even ifthe threadis in aSuspended state.
those 5 threads are just waiting running the A's and TBB doesn't have anymore threads to execute the B's...
[SergeyK] That looks very strange and I didn't experience something like this in my codes and test-cases.
If I called a 'task_scheduler_int( 255 )' a maxnumber of threads created would be 255.
Is this a possible case? In Windows Task Manager as the number of threads does it show all the TBB worker threads as well?
[SergeyK] If a thread is created it must be shown in the Task Manager even ifthe threadis in aSuspended state.
Best regards,
Sergey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
"That looks very strange and I didn't experience something like this in my codes and test-cases."
I think you are misinterpreting the described situation.
I think you are misinterpreting the described situation.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
That "pushed on to the task scheduler" is using tbb::task::enqueue(). A's are executed using the spawn, B's are executed using enqueue().
I need the waits for task dependencies. When using task::enqueue() I didn't see a way to define task dependencies using tbb API. Therefore I had to implement my own wait in the tasks.
Is TBB using lazy thread creation? If it is, when exactly does it create a thread? Because from TBB's point of view the threads running A's are not free, so it has to create more threads for B's right - given that I initialize it with a big enough MAX_THREAD number.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Make the enqueued tasks children of the tasks that want to wait for them, and use normal wait_for_all. You'll have those existing threads already to execute the enqueued tasks, and if my presumption is correct the wait calls will create additional workers. Never second-guess the scheduler by using wait mechanisms it is not aware of (unless you know perfectly well what's happening).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting Raf Schietekat
"That looks very strange and I didn't experience something like this in my codes and test-cases."
I think you are misinterpreting the described situation.
I think you are misinterpreting the described situation.
No, I'm not. A test-case I used to verify it is 4-month-old andit works. Do you have your own test-case?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
"No, I'm not. A test-case I used to verify it is 4-month-old andit works. Do you have your own test-case?"
I'm sure it does, as it also would on the setup with the reported problem, but this seems to be a different situation, because now TBB is not being asked to wait for some tasks to complete, which is where I presume the lazy thread creation occurs as a not altogether implausible explanation that fits the symptoms. Anybody who's interested is invited to test and/or explore the documentation and blogs, anybody who knows (TBB team?) is invited to clarify.
I'm sure it does, as it also would on the setup with the reported problem, but this seems to be a different situation, because now TBB is not being asked to wait for some tasks to complete, which is where I presume the lazy thread creation occurs as a not altogether implausible explanation that fits the symptoms. Anybody who's interested is invited to test and/or explore the documentation and blogs, anybody who knows (TBB team?) is invited to clarify.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting Raf Schietekat
...Anybody who's interested is invited to test and/or explore the documentation and blogs, anybody who knows (TBB team?) is invited to clarify.
@'dulantha_f'
Could you provide more technical details, please? What is your TBB version? Do you have a simple test-case? And, what is your current status regarding the problem?
Best regards,
Sergey
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, the problem was using waits that the scheduler was not aware of. My waits held back threads that TBB thought were usable.
It's not completely consistent though. Meaning if I initialize the scheduler with 5 threads and queued 5 tasks and they waited for other tasks to finish sometimes the program would run, and sometimes they won't. Mind the 5 tasks are the first 5 tasks TBB is supposed to execute. I printed out thread_ID's and I've seen 6 ID's - the only way to explain what's going on. Have to run more tests to pin point what's going on.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting dulantha_f
Yes, the problem was using waits that the scheduler was not aware of. My waits held back threads that TBB thought were usable.
It's not completely consistent though. Meaning if I initialize the scheduler with 5 threads and queued 5 tasks and they waited for other tasks to finish sometimes the program would run, and sometimes they won't. Mind the 5 tasks are the first 5 tasks TBB is supposed to execute. I printed out thread_ID's and I've seen 6 ID's - the only way to explain what's going on. Have to run more tests to pin point what's going on.
Thank you for the update.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
"I printed out thread_ID's and I've seen 6 ID's - the only way to explain what's going on."
If you ask for single-threaded operation (TBB encourages optional parallelism without mandatory concurrency partly to allow single-threaded debugging), an extra thread will be launched for any enqueued tasks (hmm, if you make an exception about non-mandatory concurrency for enqueued tasks, is two threads always enough, because otherwise why couldn't a single enqueued task also simply be scheduled between other tasks on a single thread?). Perhaps if you initialise with 5 threads and the first thread enqueues a task before a second thread has had a chance to launch, an enqueue-task thread will be launched because of how the mechanism described above is implemented but is not counted in the total? While that could be an explanation (while still in the wild-guessing stage), it would be suboptimal in the long run, so maybe it should then be corrected. But I'm not going to investigate that myself at this time, although I would be interested to hear about additional evidence (and clarification from the TBB team).
So far it would seem that a single enqueue-related thread might be launched and "forgotten", but later on additional threads are not launched for enqueued tasks except during scheduler waiting even though spawned tasks immediately get new threads. Weird and wondrous... could it really be true?
"Have to run more tests to pin point what's going on."
A self-contained test to reproduce this would be very interesting.
(Edited)
If you ask for single-threaded operation (TBB encourages optional parallelism without mandatory concurrency partly to allow single-threaded debugging), an extra thread will be launched for any enqueued tasks (hmm, if you make an exception about non-mandatory concurrency for enqueued tasks, is two threads always enough, because otherwise why couldn't a single enqueued task also simply be scheduled between other tasks on a single thread?). Perhaps if you initialise with 5 threads and the first thread enqueues a task before a second thread has had a chance to launch, an enqueue-task thread will be launched because of how the mechanism described above is implemented but is not counted in the total? While that could be an explanation (while still in the wild-guessing stage), it would be suboptimal in the long run, so maybe it should then be corrected. But I'm not going to investigate that myself at this time, although I would be interested to hear about additional evidence (and clarification from the TBB team).
So far it would seem that a single enqueue-related thread might be launched and "forgotten", but later on additional threads are not launched for enqueued tasks except during scheduler waiting even though spawned tasks immediately get new threads. Weird and wondrous... could it really be true?
"Have to run more tests to pin point what's going on."
A self-contained test to reproduce this would be very interesting.
(Edited)
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting Raf Schietekat
So far it would seem that a single enqueue-related thread might be launched and "forgotten", but later on additional threads are not launched for enqueued tasks except during scheduler waiting even though spawned tasks immediately get new threads. Weird and wondrous... could it really be true?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Maybe you are using "sleep" to alternate beteween the threads. Try to create a ThreadManage with the logic to create and balance the threads.

Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page