- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I had a program (see far-too-long post here), which I parallelized using a TBB::parallel_for loop. Each task had to do some disk reading, then some computation, but the tasks could be executed in any order. Compared to the sequential execution of the same program, this version got really bad performance out of the disk.
Later, based on the example article, I re-built the app using the TBB::pipeline, and got good disk performance back by enforcing the sequential read order, while allowing computation to proceed in parallel.
Are there other applications being developed out there that have to deal with IO and TBB? If so, what have they done to squeeze good performance out of the file system, while keeping many processors busy? Is large amounts of file read/writing common in applications that are amenable to parallel computation? I'm looking for descriptions of similar experiences with TBB and IO.
Later, based on the example article, I re-built the app using the TBB::pipeline, and got good disk performance back by enforcing the sequential read order, while allowing computation to proceed in parallel.
Are there other applications being developed out there that have to deal with IO and TBB? If so, what have they done to squeeze good performance out of the file system, while keeping many processors busy? Is large amounts of file read/writing common in applications that are amenable to parallel computation? I'm looking for descriptions of similar experiences with TBB and IO.
Link Copied
5 Replies
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Right now I'm using boost.asio, and I'm satisified with it thus far. I haven't done anything to integrate it with TBB tasks or anything yet, but I do like the model asio uses. It appears to be fairly scalable, and looks like it woud integrate well with TBB.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Quoting - AJ
Right now I'm using boost.asio, and I'm satisified with it thus far. I haven't done anything to integrate it with TBB tasks or anything yet, but I do like the model asio uses. It appears to be fairly scalable, and looks like it woud integrate well with TBB.
I totally second this "pattern". In my apps I use Boost.asio for all IO and TBB for parallel processing. As they say choose the right tool for the right job. TBB was not created with IO as a core factor, and as such it's performance and algorithms basically depend on non-blocking behavior to squeeze out the best of TBB, thus TBB+IO don't really mix well, but I guess you can shoehorn code into anything, not that it's worth it :)
Anyways nowadays I really appreciate TBB didn't cater to IO, that would have just watered it down IMHO.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Right, parallel I/O is something that will always come up... I myself wanted it for a long time, until I realized boost.asio was perfect for me. What would be very cool is if TBB could learn from mistakes that boost.asio made, polish and improve it, and incorporate it into the core library as well. That being said, I haven't done any real benchmarks on boost.asio to see how it scales.
By the way, I'm not suggesting that TBB would ever want to incorporate a lot of the other things boost.asio offers, such as DNS queries, IP addresses, sockets, and all that stuff... rather just the core pattern that it uses for parallel I/O. For those interested, boost.asio uses the proactor pattern.
Since TBB already offers tbb_thread, I'm not sure how difficult it would be to create your own boost.asio using just that.
By the way, I'm not suggesting that TBB would ever want to incorporate a lot of the other things boost.asio offers, such as DNS queries, IP addresses, sockets, and all that stuff... rather just the core pattern that it uses for parallel I/O. For those interested, boost.asio uses the proactor pattern.
Since TBB already offers tbb_thread, I'm not sure how difficult it would be to create your own boost.asio using just that.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm curious myself. Could you provide some references on boost.asio?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Reply
Topic Options
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page