Pipeline Misunderstanding?

[vetroxl]

Hi, I think I'm missing something and hopping someone can clear it up for me? My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed? Here is what I'm trying to do

First-stage = serial_in_order, contains a circular vector conatiner, recycles when end is reached
Second-stage = parallel, and process theelement handed by the first stage
Last-stage = serial_in_order, does nothing, just used asto synchro

What I'm seing is that the first stage gets initiated before the last token has completed? I had to add a condition variable to wait for the vector element to be "ready". What am I missing? My number of tokens = the number of elements in my vector. I put an assertion to verify this.

Thanks.

[RafSchietekat · Accepted solution]

In this configuration, the input filter can run ahead of the output filter modulo the size of the buffer, but it shouldn't catch up with any item still in flight. There's an example that does exactly this.

"My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed?"
Two serial_in_order filters will process data item in the same order, and there's a maximum number of items in flight, but that's it, and just as well.

"What I'm seing is that the first stage gets initiated before the last token has completed?"
If you mean that the first stage starts on a new item before the previous item has completely traversed the pipeline, the opposite would preclude parallel operation, so it's a good thing.

[vetroxl]

Quoting - Raf Schietekat

In this configuration, the input filter can run ahead of the output filter modulo the size of the buffer, but it shouldn't catch up with any item still in flight. There's an example that does exactly this.

"My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed?"
Two serial_in_order filters will process data item in the same order, and there's a maximum number of items in flight, but that's it, and just as well.

"What I'm seing is that the first stage gets initiated before the last token has completed?"
If you mean that the first stage starts on a new item before the previous item has completely traversed the pipeline, the opposite would preclude parallel operation, so it's a good thing.

Ok, I know where my race condition is, I just thought by putting the last sage as serial in order it would auto synchronize...thanks for your help.

[RafSchietekat]

It might have been instructive to read what the problem was, exactly...

[vetroxl]

Quoting - vetroxl

Hi, I think I'm missing something and hopping someone can clear it up for me? My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed? Here is what I'm trying to do

First-stage = serial_in_order, contains a circular vector conatiner, recycles when end is reached
Second-stage = parallel, and process theelement handed by the first stage
Last-stage = serial_in_order, does nothing, just used asto synchro

What I'm seing is that the first stage gets initiated before the last token has completed? I had to add a condition variable to wait for the vector element to be "ready". What am I missing? My number of tokens = the number of elements in my vector. I put an assertion to verify this.

Thanks.

So if this is all true, then my buffer should never be empty when the first stage, starts correct?

[RafSchietekat]

"So if this is all true, then my buffer should never be empty when the first stage, starts correct?"
It should guarantee that the input filter is only invoked with a free slot. Obviously initially the circular buffer will be empty, and it could become empty again during processing, but the input filter is not invoked while the circular buffer is already full.

[vetroxl]

Quoting - Raf Schietekat

"So if this is all true, then my buffer should never be empty when the first stage, starts correct?"
It should guarantee that the input filter is only invoked with a free slot. Obviously initially the circular buffer will be empty, and it could become empty again during processing, but the input filter is not invoked while the circular buffer is already full.

Thanks! I managed to find the bug in my code. It was a "zero based" indexing issue and not a race condition. Kind of embarassing actually :)