Thread to Core Mapping

[irisshinra308]

Hi~

I have a question about the thread to core mapping in the TBB.

As far as I know, each TBB thread should be binded on a specific core to make sure the reusability of the affinity data.

Could anybody tell me what the mapping is?

Thanks for reading

Dennis

[RafSchietekat]

My understanding is that TBB finds out how many threads can comfortably run on a particular machine, i.e., without oversubscription, sets up the total number of threads to be that number by default, and lets the operating system figure out the rest. Also, unless I'm mistaken, it's not exactly the number of cores that matters, but the number of hardware threads, so, e.g.,one core and 2 hyperthreads would still give you two TBB threads.

[irisshinra308]

Thanks for replying!!

Sorry for the confusing description on my problem above.

I want to know that if TBB binds threads onto specific core or hardware context.

That is, Would TBB try to prevent TBB threads from migrating from core to core in execution?

I know this could be done in Pthread by calling the pthread_setaffinity_np().

The program could use this method to bind threads on a specific core to maintain the thread affinity.

In TBB, we could specify the affinity id of tasks, so I think there might be a similar mapping between threads and cores to maintain the task affinity property.

If the mapping do exist in TBB, I would like to know what the mapping is.

Thanks for reading this.

Any help would be grateful!!

Dennis

[Alexey-Kukanov]

Quoting - irisshinra308

I want to know that if TBB binds threads onto specific core or hardware context.

That is, Would TBB try to prevent TBB threads from migrating from core to core in execution?

I know this could be done in Pthread by calling the pthread_setaffinity_np().

The program could use this method to bind threads on a specific core to maintain the thread affinity.

In TBB, we could specify the affinity id of tasks, so I think there might be a similar mapping between threads and cores to maintain the task affinity property.

TBB does not bind threads to cores. We believe this is the job of operating system to schedule threads in the best possible way, given current state of the system at whole. And we assume OS does its best to re-bind threads to the same cores where they run before, especially if the system is not oversubscribed. The affinity mechanism in TBB maps tasks to threads, in the above mentioned assumption. Meanwhile, in many cases TBB uses internal mechanisms (such as scheduler bypassing, stealing rules, etc) to ensure good cache locality.

As far as I know, no user-specifed binding (affinity masks and etc.) can guarantee thread will never migrate. OS still leaves itself some freedom of assigning a thread to a non-specified core, at least temporarily.

[Dmitry_Vyukov]

Quoting - irisshinra308

As far as I know, each TBB thread should be binded on a specific core to make sure the reusability of the affinity data.

When I was looking at the sources last time there was no binding of threads to cores at all. Why do you think that there is such binding?

[Dmitry_Vyukov]

Quoting - Alexey Kukanov (Intel)

As far as I know, no user-specifed binding (affinity masks and etc.) can guarantee thread will never migrate. OS still leaves itself some freedom of assigning a thread to a non-specified core, at least temporarily.

Binding provided by POSIX and Win32 API is STRICT, which means that on return from the function call target thread can not run on the non-specified processors. Though not sure what happens when processor goes down or when host OS changes number of VM processors...

[irisshinra308]

Quoting - Dmitriy Vyukov

When I was looking at the sources last time there was no binding of threads to cores at all. Why do you think that there is such binding?

I have seen some pthread application using the pthread_setaffinity_np() to bind thread onto a specific core to ensure the cache affinity.

By doing so, the programmer could be sure about that the data in the cache which was taken from the previous thread could be reused by the following thread.

Some application shows significant performance improvement by maintain the cache affinity.

Because programmer could specify the affinity_id to hint the scheduler, I think there might be some similar mapping to ensure to cache affinity.

[RafSchietekat]

While it is not impossible that the operating system would disturb TBB's long-lived threads, the assumption is that they stay put long enough that it is sufficient to identify task affinity to threads to achieve cache locality. Only if you want to directly manipulate your own threads (a less efficient way to use the parallel hardware than task-based processing), would you need to consider how those threads are bound to cores. Rest assured that cache-locality concerns were a high priority in the design of TBB, even if the details work out a little differently.