Pipeline where a filter generates multiple tokens -- possible?

[jind]

Hi,

Our application looks like A->B->C, where the result of B can be either (1) nothing further to process, or (2) a number of new things to process. I'm new to TBB, but it seems like the proper thing in case (1) is to return NULL and have C do nothing on NULLs. Case 2 is harder, though, since B (as a tbb:filter) is constrained to return a single token (void*).

The only solution I can see is to have B return a pointer to a set of tokens. But then I will need to handle the parallelism of C internally (to the filter), as opposed to letting the pipeline handle it, which would seem more natural.

Is there something I'm missing here?I'm somewhat new to this, so please pardon any obvious oversights...

Thanks,

John

[Alexey-Kukanov · Accepted solution]

These use cases are not supported by the TBB pipeline. I think that returning NULL from the middle stage might also be unsafe. In the presense of in-order stages, it is hard to support such semantics because it may break the establishedorder by either removing some tokens or introducing new ones, both leading to problems.

My recommendation is to always pass on some structure containing the information about further processing requirements, parse it in stage C, and process as appropriate. C can utilize other TBB constructs (such as parallel_invoke or task_group, bothavailable in the recent open-source updates and targeting the next commercial release, but also good old parallel_for etc.) as appropriate to parallelize independent processing ofmultiple real tokens passed from B at once.