Would it be possible to execute a Producer Consumer program using tasks? Or would pipelines be more appropriate? I've attached a flowchart of the kind of Producer Consumer problem I'm talking about. In the flowchart, the input section (left side) collects a lot of inputs for a fixed number of seconds and then intimates the processing section (middle) which processes the inputs and when the processing is done, the output section (right side) is intimated.
While the processing section is busy processing the data, the input section should be able to continue taking new inputs from the ports. The output section should also be able to work parallelly.
My questions are: 1. Is the solution to such a problem best done using tasks or pipelines? 2. Is there a way to keep I/O threads separate from data processing threads (I've read in Intel's parallel programming book that this improves performance significantly) 3. If I use tasks or pipelines, will my program be independent of the number of threads? (as I understand, each task would be assigned a thread. But I'd prefer to have a logic where any available thread of the thread pool would be able to execute a waiting task)
I think pipeline with serial-parallel-serial filters should be an optimal solution. Note that you can safely use any parallel algorithms in the middle (processing) filter. And if the data on the input stage are accumulated fast enough, you'll have several data batches processed concurrently by the middle stage. This way you can utilize all available CPUs.
Unfortunately TBB does not handle well IO operation so far because of the lack of support on the OS side. Yet to compensate for waiting inside IO calls you could try to instantiate TBB sceduler oversubscribed by one thread.