TBB Thread priority

[jogshy]

Hello,

We're making a multiplatform(win,linux,macosx,solaris) software renderer. It uses a lot of CPU power and I need use low priority threads so the user could make another thing ( for instance, to browse the Internet or to open Photoshop ) while the image is computing.

AFAIK, OpenMP does not allow to set manually the thread priority... although it's possible to change on the fly the thread priority using nasty unportable tricks that almost all the times don't event work properly... ( yes, we're using OpenMP currently)

I think the current TBB's version neither can set the working thread's priority manually... But if this could be possible then TBB would be the perfect solution for us! Also would be nice to set manually the CPU affinity ( = I want this task running only in 3 cores instead of the 4 of my quad CPU ).

As I could intuit the answer ( no!, you cannot! use OS threads! )... let me say that we use multithread almost for every thing in our application... from tone mapping, passing by mesh subdivision and finishing by raytracing... and to implement this using pthreads/Win32 threads is just and plain unacceptable ( a nightmare, in fact ).

thx

[Dmitry_Vyukov · Accepted solution]

TBB allows you to execute some arbitrary "initialization" code for every worker thread, you can do this by using task_scheduler_observer interface.
However, AFAIK, TBB does not provide portable interface for setting thread priorities and affinity. So you will have to do this manually with something like:
SetCurrentThreadPriority(GetCurrentThread(), THREAD_PRIO_LOW); // or pthread analog
Regarding usage of only 3 cores out of 4, when you initialize TBB task scheduler you may override a number of worker threads.
So you may instruct TBB to use only 3 threads, then with the help of task_scheduler_observer set required affinity and priority for each thread.

[jogshy]

Thanks Dimitriy, the observer will work.... but I'm gonna need to plague the code by zillions of #ifdef linux elseif Windows elseif MacOSX elseif Solaris elseif... blah blah. and call the native OS functions to change the priority and affinity... It will work, yes, but it's far from perfect.

I think Intel should add some method to control this easily. It won't be hard to add a second parameter to the task_scheduler_init ( #threads, desiredWorkerPriority, std::vector cpuCoreAffinity ). That's a way better to perform that, because some platforms don't allow to change the priority after the threads are created.

[Dmitry_Vyukov]

Quoting - jogshy

Thanks Dimitriy, the observer will work.... but I'm gonna need to plague the code by zillions of #ifdef linux elseif Windows elseif MacOSX elseif Solaris elseif... blah blah. and call the native OS functions to change the priority and affinity... It will work, yes, but it's far from perfect.

Humm... I would prefer just 1 #ifdef instead of zillions.

#ifdef _WIN32

void set_current_thread_priority()
{
SetThreadPriority(GetCurrentThread(), THREAD_PRIO_LOW);
}

void set_current_thread_affinity(int proc)
{
SetThreadAffinityMask(GetCurrentThread(), 1 << proc);
}

#elif _POSIX

void set_current_thread_priority()
{
pthread_...
}

void set_current_thread_affinity(int proc)
{
pthread_...
}

#endif

Then use set_current_thread_priority() and set_current_thread_affinity() in the code.

[Dmitry_Vyukov]

Quoting - jogshy

I think Intel should add some method to control this easily. It won't be hard to add a second parameter to the task_scheduler_init ( #threads, desiredWorkerPriority, std::vector cpuCoreAffinity ). That's a way better to perform that, because some platforms don't allow to change the priority after the threads are created.

If it is considered as a feature request by TBB team, then I vote for it too. I have exactly the same situation (I do not use TBB, though): parallelized long-running computations that better run in background and do not disturb end user. So I just setup low priority for all worker threads at startup.

[jogshy]

Btw...

why I should deal with Win32, pthreads, etc functions? Is not there a tbb_thread::SetPriority() method? It should!

[Alexey-Kukanov]

Quoting - jogshy

Btw...

why I should deal with Win32, pthreads, etc functions? Is not there a tbb_thread::SetPriority() method? It should!

The API of tbb_thread was copied from the std::thread class in upcoming C++0x, with necessary adaptation to the current C++ standard. And as you might guess, std::thread does not support priorities (yet). We decided that the C++ committee put enough consideration on the thread class, and so there must be some reasons to not add API for priorities for the moment. Adding yet another non-standard API to the existing mix did not seem wise.

[RafSchietekat]

"We decided that the C++ committee put enough consideration on the thread class, and so there must be some reasons to not add API for priorities for the moment."
Perhaps a bit naive. Instead of cleaning up the folly that equality of associative containers is based on the element's == and < operators (making equality etc. based on order of insertion!!!), now you will also have to be aware that, between, say, multiset and unordered_multiset,the former considers order of insertion (or does it? where is it defined in what order equivalent keys are stored?) and the latter doesn't (but still looks at element::operator== to decide about equality instead of staying within the container's logic). All because the original specification was so very much in love with the brevity of one definition for all containers (based on std::equal for an iteration, which really only makes sense for sequential containers), whether it made sense or not, and with the brevity of reusing std::pair::operator== in maps, again whether it made sense or not. For clarity, my advice would be to deprecate comparison of ordered associative containers (that one is beyond redemption for two separate reasons, but redefining it would silently break existing code), and to use the container's equality function for comparison of unordered (=hash) associative containers.

[Dmitry_Vyukov]

Quoting - Alexey Kukanov (Intel)

The API of tbb_thread was copied from the std::thread class in upcoming C++0x, with necessary adaptation to the current C++ standard. And as you might guess, std::thread does not support priorities (yet). We decided that the C++ committee put enough consideration on the thread class, and so there must be some reasons to not add API for priorities for the moment. Adding yet another non-standard API to the existing mix did not seem wise.

Alexey, I do not agree with you here.
There are a lot of things that are missed in the current draft by some reason, for example rw_mutex, don't you think that it's a kind of "must be" for a threading library? (I guess you do because it's present in the TBB). Also, there is no standard way to do blocking in lock-free algorithms. And their tasking API is quite primitive.
As far as I understand, the reasons for that are (1) requirement to provide exhaustive formal semantics, (2) limited time. Useful formal specification of thread priorities is quite tricky, and they also have to specify all possible interactions with other features. And for TBB there is no such problems... ok, at least they are not so... problematic.

Ok, if the things are the way you present them, then why they ever need following method?
native_handle_type thread::native_handle();

And the second question, why most OSes do provide thread priorities and affinities in their APIs?

I believe that the way to go for a library like TBB is to go beyond the standard and provide things like priorities and affinities as extensions to standard API.
If you wish to keep tbb_thread interface as close to std as possible, then you may provide extended methods as a free functions (and probably specifically mark them as non-standard - "_ns"). Something along the lines of:
void tbb::set_thread_priority_ns(tbb::tbb_thread& t, tbb::thread_prio_ns prio);
void tbb::set_thread_affinity_mask_ns(tbb::tbb_thread& t, tbb::thread_affinity_mask_ns mask);

[bvj]

Quoting - Dmitriy Vyukov

I believe that the way to go for a library like TBB is to go beyond the standard and provide things like priorities and affinities as extensions to standard API.

Exactly.

Life is too short to *wait* on the standard. I vote for preemptive implementation of modern thread properties in TBB.

The inclusion of regular thread support in TBB is greatly appreciated. I don't see it interefering with the purity of TBB's parallelism support.

From both the design and implementation standpoint, I prefer not to scatter threading and synchronization across multiple libraries.