Per Thread Init?

[michaelnikelsky]

Hi,

is there a way to have a init function that is called for every thread the task scheduler creates? I am trying to use TBB with OpenSG which has something called aspects where each thread gets its own aspect. However, this also requires me to set the correct aspect for each thread to work with. Calling this in the operator() is not really an option since it causes massive slowdowns.

The only idea I came up with so far was to create a dummy parallel-for that does nothing else but initilizing a thread but I am not shure how I can assure that every thread is used for the parallel-for.

Does anybody have an idea? Maybe for a future version of tbb the task_scheduler could take a callback-function that is called for each thread during initilization?!

Michael

[ARCH_R_Intel]

You are absolutely right, there is currently no way to guarantee that you have detected every thread. We did a good job of encouraging task-based instead of thread-based programming J. Seriously, the TBB group has kicked around notions about having a mechanism that notifies clients when it is creating or destroying a thread, but we had only one use case to motivate us. You've provided a second, so clearly we should address this now, while were dealing with the stack size parameter discussed elsewhere.

So lets discuss the design of this feature so people can chime in.

It seems the right solution is to add some virtual functions to task_scheduler_init, and let clients override them. Virtual functions provide a level of type safety lacking from C++ callbacks.

There are some interesting choices in the design space.At one extreme, we could have the virtual functions do the actual thread creation and destruction. Then clients could add all the hooks they want. But thats insanely general, because a lot of burden would be put on clients to get the details right, and the code would be very OS specific. Anyone who really needs that level of generality should go hack the TBB sources.

A less general, but saner approach would be to define two virtual functions for task scheduler init:

// Called immediately after TBB creates a thread, by the new thread.

virtual void note_thread_creation() {}

// Called immediately before a TBB destroys a thread, by the dying thread.

virtual void note_thread_destruction() {}

In principle, note_thread_destruction is not necessary because most threading packages provide a way to detect when a thread exits, but it seems nice to provide it here for symmetry and provide notification without requiring clients to get into OS details.

Would this be enough support for most clients, or is it missing some key ingredient?

- Arch

P.S. One limitation that this design would put on us is that we would have to guarantee that all threads created by TBB call note_thread_destruction() before ~task_scheduler_init runs, because otherwise those calls are on a destroyed object. But I think this limitation is reasonable, and already there in the scheduler for another reason. In an early prototype of TBB, we let ~task_scheduler_init return before all worker threads were destroyed, and run into problems on some OSes, where a tight loop like this:

for() {

 task_scheduler_init init;

 ...

}

would let threads pile up, because some OSes would let new threads be created faster than the old threads were being destroyed. Hence t he current TBB implementation already constrains ~task_scheduler_init to wait, in order to avoid the pileup.

[ARCH_R_Intel]

I forgot to mention that most of TBB avoids virtual functions in favor of compile-time polymorphism. But thread creation/destruction is so heavyweight anyway that two virtual calls doesn't matter.

[michaelnikelsky]

Thanks for your reply. Having a virtual function sounds like a good idea. This function should at least receive the thread-id of the created thread, so it can do some initializations to this thread.

[ARCH_R_Intel]

Passing a thread id introduces anOS-specific type into the TBB interface. So far, we've avoided that. Indeed, only one of our public headers even mentions pthreads.

Thus I'm inclined to not pass the thread id, but instead depend upon the user to fetch it themselves. E.g, for pthreads, use pthread_self() and for Win32, use GetCurrentThread(). I realize this is a bit of a contortion, having an interface not mention OS-specific thread ids, though it is intended to do thread-specific work.