Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.
2464 Discussions

Using both, Threads and Tasks. Does it make sense?

erik_rostock
Beginner
280 Views

Hello,

I am not very familiar with Intel TBB, but while reading a bit in the docs and looking at some presentations I was wondering whether it makes sense to use both, Threads and Tasks.

The background is the following: I am thinking of an internet TCP server that accepts several connections. To handle input I would start a new thread for each client in order not to block the main loop with any networks troubles. For any other actions I would create tasks.

Maybe I should add the information that the server application is thought to run on normal home PC's, having 2 or 4 cores.

I hope you can help me - thanks for your answers!

Erik

0 Kudos
6 Replies
Dmitry_Vyukov
Valued Contributor I
280 Views
Quoting - erik.rostock

Hello,

I am not very familiar with Intel TBB, but while reading a bit in the docs and looking at some presentations I was wondering whether it makes sense to use both, Threads and Tasks.

The background is the following: I am thinking of an internet TCP server that accepts several connections. To handle input I would start a new thread for each client in order not to block the main loop with any networks troubles. For any other actions I would create tasks.

Maybe I should add the information that the server application is thought to run on normal home PC's, having 2 or 4 cores.

I hope you can help me - thanks for your answers!

What profit are you expecting to get from such combination?

Probably, there are situations when threads+tasks makes sense. For example, server that accepts one *heavy* request in a few seconds. Server runs on a hardware with substantial number of hardware threads (not just 2 cores, let's say 8 cores). Server can use threads to manage tcp-connections (just because that's simpler) and task to process requests (because requests are heavy, i.e. parallelization is worth doing; and there is no substantial parallelism on the request level, i.e. one request in a few seconds).

But if server accepts many requests per second and requests are relatively light (for example, request takes ~10ms), then intra-request parallelization is silly idea. In such situation parallelization must be done on request level. Hint: choose highest possible level for parallelization.

0 Kudos
robert_jay_gould
Beginner
280 Views
Quoting - erik.rostock

Hello,

I am not very familiar with Intel TBB, but while reading a bit in the docs and looking at some presentations I was wondering whether it makes sense to use both, Threads and Tasks.

The background is the following: I am thinking of an internet TCP server that accepts several connections. To handle input I would start a new thread for each client in order not to block the main loop with any networks troubles. For any other actions I would create tasks.

Maybe I should add the information that the server application is thought to run on normal home PC's, having 2 or 4 cores.

I hope you can help me - thanks for your answers!

Erik

Tbb is great for multithreading tasks, threads in general are useful for IO.

However from experience, and my own situation right now, I use Boost::Asio for all my IO (the performance is about as good as handcoding the whole thing, its multi-platform and you can get a generally bug-free (I don't believe in true bug-freedom), portable server system running within an hour, while coding your own scalable, platform agnostic system using modern C++ style, and 90% correct IO system will take months, and if your really good it might out perform Boost::Asio.

So my recommendation is use Asio for IO and tbb for logic. It works great for me.

0 Kudos
erik_rostock
Beginner
280 Views
Quoting - Dmitriy V'jukov

What profit are you expecting to get from such combination?

Probably, there are situations when threads+tasks makes sense. For example, server that accepts one *heavy* request in a few seconds. Server runs on a hardware with substantial number of hardware threads (not just 2 cores, let's say 8 cores). Server can use threads to manage tcp-connections (just because that's simpler) and task to process requests (because requests are heavy, i.e. parallelization is worth doing; and there is no substantial parallelism on the request level, i.e. one request in a few seconds).

But if server accepts many requests per second and requests are relatively light (for example, request takes ~10ms), then intra-request parallelization is silly idea. In such situation parallelization must be done on request level. Hint: choose highest possible level for parallelization.

The server will handle not too much new requests, thats all done at the beginning. If all the connections are made I have really a high load of packages being sent, ~ 15/s to 30/s. I don't want that the receiving and sending of the packets is done in the main loop. Maybe I could start a thread that handles network input and define tasks to process it.

0 Kudos
Dmitry_Vyukov
Valued Contributor I
280 Views
Quoting - erik.rostock

The server will handle not too much new requests, thats all done at the beginning. If all the connections are made I have really a high load of packages being sent, ~ 15/s to 30/s. I don't want that the receiving and sending of the packets is done in the main loop. Maybe I could start a thread that handles network input and define tasks to process it.

If you will create thread per connection, then with high probability you don't need any tasks. Just make all the processing in those threads.

If you will have single thread for IO, then you need tasks to utilize all cores. But probably the best approach will be to create just single task per 'package'.

0 Kudos
Alexey-Kukanov
Employee
280 Views
Quoting - Dmitriy V'jukov
If you will create thread per connection, then with high probability you don't need any tasks. Just make all the processing in those threads.

If you will have single thread for IO, then you need tasks to utilize all cores. But probably the best approach will be to create just single task per 'package'.

I'd say that depend on the number of connections. If there is quite a lot of those, such as an order of magnitude more than the number of cores, then a thread per connection does not sound quite right.

I'd probably have a dedicated thread for IO, and a separate thread that would process packages; this thread would use TBB to parallelize the processing. The granularity would possibly be smaller than a task per package because30 packages per second sounds too coarse-grained (I assume there is some work in a package to keep the cores busy for about 10-30 milliseconds, otherwise there would be not much need in those cores).

0 Kudos
Dmitry_Vyukov
Valued Contributor I
280 Views

I'd probably have a dedicated thread for IO, and a separate thread that would process packages; this thread would use TBB to parallelize the processing. The granularity would possibly be smaller than a task per package because30 packages per second sounds too coarse-grained (I assume there is some work in a package to keep the cores busy for about 10-30 milliseconds, otherwise there would be not much need in those cores).

I would recomment to start with task per package, because it's simplier and requires less coding/testing. Although it depends on requirements. But note that in many situations dedicated core can process package very quickly. For example, with 1 core package processing takes 1 ms, with 4 cores it will take 1/4 ms - if the difference makes sufficient sense in your context then you can consider intra-package parallelization. And you can always add intra-package parallelization later.

And also note that you better choose the highest possible level for parallelization (although the meaning of "possible" depends on the context).

0 Kudos
Reply