Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.
2442 Discussions

flow_graph large data handling/passing - best practice ?

nyue
Novice
255 Views

I am considering splitting up the various task I currently have with geometry processing of point-cloud/lidar-scan data and rewrite/wrap them as tasks and connect them up via flow_graph.

 

As the dataset will not be small, I plan to pass them between the flow_graph nodes via pointer/smart-pointer.

 

If the node modify the data, the node is responsible for duplicating (and registering it with some singleton data manager of some sort which I have to figure out and implement) the data before modifying and passing them on downstream.

 

Are there threading/concurrency aspect that I should be mindful of ? Is the above approach feasible ?

 

I believe these are common problems that many people have encountered in the past, are there any write up about such things ?

 

Kind regards

0 Kudos
1 Solution
SeshaP_Intel
Moderator
227 Views

Hi,


Thank you for posting in Intel Communities.


The flow graph passes data structures from node to node. The messages can be primitive types, objects, and pointers. You can use pointers as messages.


You can use buffering nodes like buffer_node or write_once_node which forward the message to successor. Please refer to the below links for more details.

https://spec.oneapi.io/versions/latest/elements/oneTBB/source/flow_graph/buffer_node_cls.html

https://spec.oneapi.io/versions/latest/elements/oneTBB/source/flow_graph/write_once_node_cls.html


You can use the concurrency limits on nodes to limit resource consumption. Please refer to the below link for more details.

https://oneapi-src.github.io/oneTBB/main/tbb_userguide/Flow_Graph_resource_tips.html


Thanks and Regards,

Pendyala Sesha Srinivas


View solution in original post

2 Replies
SeshaP_Intel
Moderator
228 Views

Hi,


Thank you for posting in Intel Communities.


The flow graph passes data structures from node to node. The messages can be primitive types, objects, and pointers. You can use pointers as messages.


You can use buffering nodes like buffer_node or write_once_node which forward the message to successor. Please refer to the below links for more details.

https://spec.oneapi.io/versions/latest/elements/oneTBB/source/flow_graph/buffer_node_cls.html

https://spec.oneapi.io/versions/latest/elements/oneTBB/source/flow_graph/write_once_node_cls.html


You can use the concurrency limits on nodes to limit resource consumption. Please refer to the below link for more details.

https://oneapi-src.github.io/oneTBB/main/tbb_userguide/Flow_Graph_resource_tips.html


Thanks and Regards,

Pendyala Sesha Srinivas


SeshaP_Intel
Moderator
71 Views

Hi,


Thanks for accepting our solution. If you need any additional information, please post a new question as this thread will no longer be monitored by Intel.


Thanks and Regards,

Pendyala Sesha Srinivas


Reply