Community
cancel
Showing results for 
Search instead for 
Did you mean: 
vetroxl
Beginner
49 Views

Pipeline Misunderstanding?

Jump to solution
Hi, I think I'm missing something and hopping someone can clear it up for me? My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed? Here is what I'm trying to do

First-stage = serial_in_order, contains a circular vector conatiner, recycles when end is reached
Second-stage = parallel, and process theelement handed by the first stage
Last-stage = serial_in_order, does nothing, just used asto synchro

What I'm seing is that the first stage gets initiated before the last token has completed? I had to add a condition variable to wait for the vector element to be "ready". What am I missing? My number of tokens = the number of elements in my vector. I put an assertion to verify this.

Thanks.
0 Kudos
1 Solution
RafSchietekat
Black Belt
49 Views
In this configuration, the input filter can run ahead of the output filter modulo the size of the buffer, but it shouldn't catch up with any item still in flight. There's an example that does exactly this.

"My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed?"
Two serial_in_order filters will process data item in the same order, and there's a maximum number of items in flight, but that's it, and just as well.

"What I'm seing is that the first stage gets initiated before the last token has completed?"
If you mean that the first stage starts on a new item before the previous item has completely traversed the pipeline, the opposite would preclude parallel operation, so it's a good thing.

View solution in original post

6 Replies
RafSchietekat
Black Belt
50 Views
In this configuration, the input filter can run ahead of the output filter modulo the size of the buffer, but it shouldn't catch up with any item still in flight. There's an example that does exactly this.

"My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed?"
Two serial_in_order filters will process data item in the same order, and there's a maximum number of items in flight, but that's it, and just as well.

"What I'm seing is that the first stage gets initiated before the last token has completed?"
If you mean that the first stage starts on a new item before the previous item has completely traversed the pipeline, the opposite would preclude parallel operation, so it's a good thing.

View solution in original post

vetroxl
Beginner
49 Views
Quoting - Raf Schietekat
In this configuration, the input filter can run ahead of the output filter modulo the size of the buffer, but it shouldn't catch up with any item still in flight. There's an example that does exactly this.

"My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed?"
Two serial_in_order filters will process data item in the same order, and there's a maximum number of items in flight, but that's it, and just as well.

"What I'm seing is that the first stage gets initiated before the last token has completed?"
If you mean that the first stage starts on a new item before the previous item has completely traversed the pipeline, the opposite would preclude parallel operation, so it's a good thing.

Ok, I know where my race condition is, I just thought by putting the last sage as serial in order it would auto synchronize...thanks for your help.
RafSchietekat
Black Belt
49 Views
It might have been instructive to read what the problem was, exactly...
vetroxl
Beginner
49 Views
Quoting - vetroxl
Hi, I think I'm missing something and hopping someone can clear it up for me? My understanding is that the pipeline will not schedule any first-stage filters until the last stage filter has completed? Here is what I'm trying to do

First-stage = serial_in_order, contains a circular vector conatiner, recycles when end is reached
Second-stage = parallel, and process theelement handed by the first stage
Last-stage = serial_in_order, does nothing, just used asto synchro

What I'm seing is that the first stage gets initiated before the last token has completed? I had to add a condition variable to wait for the vector element to be "ready". What am I missing? My number of tokens = the number of elements in my vector. I put an assertion to verify this.

Thanks.

So if this is all true, then my buffer should never be empty when the first stage, starts correct?
RafSchietekat
Black Belt
49 Views
"So if this is all true, then my buffer should never be empty when the first stage, starts correct?"
It should guarantee that the input filter is only invoked with a free slot. Obviously initially the circular buffer will be empty, and it could become empty again during processing, but the input filter is not invoked while the circular buffer is already full.
vetroxl
Beginner
49 Views
Quoting - Raf Schietekat
"So if this is all true, then my buffer should never be empty when the first stage, starts correct?"
It should guarantee that the input filter is only invoked with a free slot. Obviously initially the circular buffer will be empty, and it could become empty again during processing, but the input filter is not invoked while the circular buffer is already full.

Thanks! I managed to find the bug in my code. It was a "zero based" indexing issue and not a race condition. Kind of embarassing actually :)
Reply