- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I've have a piece of toy code that runs a large number of "events" through TBB. For each event a flow graph is created with a small number of tasks, then these events are run in batches of 30, using parallel_foreach to call wait_for_all on each set of graphs.
I'm finding that as the number of events executed to date in a particular run increases (i.e. as time increases), the time taken per event also increases, and I'm at a loss as to why this might be.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looking through the code, tbb_main.cpp, it looks like that the loop, starting line 94, creates more and more TBB Flow Graphs (FGs)? Is that correct? Usually, the data is fed into the already created static FG. I guess, this sample reproduces behavior of real application but it would be nice to understand what you are trying to achieve, e.g., the architecture of application, etc. Furthermore, from your description and the code, it seems that the FGs being created are only active in batches which results in more and more inactive FGs collected in memory that somehow produces slowdown. One of the possible guesses is that we are looking at the memory leak. Have you tried to watch the memory size while running your experiments?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for reaching out to us.
Could you please provide us with a sample reproducer and the steps you have followed to reproduce the issue so that we can try it from our end?
Please let us know how you are measuring the performance of your code.
Also please provide the OS details.
Thanks & Regards,
Noorjahan
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
The code is here: https://github.com/beojan/HPXDemo/tree/master/src/events_tbb
I'm measuring the performance using the std::chrono steady_clock. The machine labeled "Laptop" is running Arch on an i7-11800H, while the one labeled Zeus is running CentOS 7 on a pair of Xeon Gold 5220s.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for providing the details.
We tried to build the code from our side but we are facing some issues.
Please find the below screenshot for the error we are facing:
We tried with Boost versions 1.66 and 1.79 but still facing the issue.
Could you please help us to reproduce your issue from our end(Commands/steps to reproduce the issue)?
Thanks & Regards,
Noorjahan.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
We also tried with Boost version 1.79 and were able to compile it successfully but we are facing issues at runtime.
Please find the below screenshot for the error we are facing:
Could you please help us to reproduce your issue from our end?
Thanks & Regards,
Noorjahan.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Try ./a.out ../../test/test.txt 16
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looking through the code, tbb_main.cpp, it looks like that the loop, starting line 94, creates more and more TBB Flow Graphs (FGs)? Is that correct? Usually, the data is fed into the already created static FG. I guess, this sample reproduces behavior of real application but it would be nice to understand what you are trying to achieve, e.g., the architecture of application, etc. Furthermore, from your description and the code, it seems that the FGs being created are only active in batches which results in more and more inactive FGs collected in memory that somehow produces slowdown. One of the possible guesses is that we are looking at the memory leak. Have you tried to watch the memory size while running your experiments?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, it's a toy example. I would prefer to use a single static flow graph, but (as far as I could tell) there seemed to be no way to tie the inputs corresponding to a single "event" (think database row) together and await the results corresponding to that event from the end of the flow graph. Is there some way to do this that I'm missing?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Looking a bit further, I was deleting the nodes but not the graphs when I was done with them. Now that I'm deleting the graphs the problem appears to be solved.
Nevertheless I would still like to know if there's a way to tie together inputs and match them to outputs so I can use a single graph instance.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I'm not sure what is specifically the issue you are facing? Could you sketch the graph you are trying to construct? Have you looked through the documentation related to TBB Flow Graph APis starting from, e.g.:
https://oneapi-src.github.io/oneTBB/main/tbb_userguide/Nodes.html
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Could you look at the join node, i.e. i) tag_matching policy joins inputs together that have matching tags and ii) a token-based system that can be created by using reserving join_nodes, here
https://oneapi-src.github.io/oneTBB/main/tbb_userguide/create_token_based_system
In particular, an example in the link above might help to understand how to construct and use a token based system. Please let us know if these examples are along the lines of the demo you are working on.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hello,
I hope that you found the last comments about join node useful. With no response from you, we won't monitor this ticket internally anymore.
![](/skins/images/D3C0B914909A6564BBB97F4AD1ED1973/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page