HyPhi, tasks with blocking IO

[Ender1618]

Has anyone worked with the HyPhi library, it supposedly modifies TBB's task scheduler to better deal with some tasks that may need to perform blocking IO operations. I can find academic papers about the library, but I can't find an implementation. Does anyone know if one exists?

If not does anyone one know how to optimally implement the same concept with TBB. I have a pipes and filters framework that currently uses a native thread per filter, where i would rather use tasks that can be managed to utilize work stealing (like the TBB scheduler does not natively). The issue is that the source filters in my graph are receiving there data via live sensors, which do no provided data at very high frequencies, so therefore some filter's need to wait for a sensor sample to be available to work with it, which is a blocking call (currently waits a condition variable to be signaled).

I am trying to find a solution that would allow some filters to block until data is available and have the TBB scheduler be able to retask that thread (the one that switch to a wait) to another parallel computation in the mean time.

HyPhi seemed to be able to do something like this.

Thx for any advice,

-Ryan

[RafSchietekat]

Have you considered thread_bound_filter for the I/O-bound stage?

[Ender1618]

I was under the understanding that the TBB pipeline framework, could only be used for a linear set of filters.  The filters in my graph can be connected in an arbitrary DAG (so splits and joins).  I may have misunderstood something there, though.

Another issue is that a filter (in my framework) may only be IO bound for a limited time. Meaning if it is feed data at the rate of the connected input filters production rate (and can consume at that rate), then it would not need to block. But this isnt always true, if the producer slows down (since it compute time for a sample could be variable dependent on its content), then the consuming filter would need to block.

Source filters (feed by sensors) can be producing data at different rates as well.

[ronag89]

See my post about "thread_bound_node" in the following topic https://software.intel.com/en-us/forums/topic/516144#comment-1790621.

Might give you some idea...