TBB and Multi-Tasking

[study24816]

I need to write a program which does 2 tasks at the same time for better efficiency & high response. First task is, for example, get vision data from a camera & process it.

Second task is, receive processed data from first task & do sth else with this data (robot control strategy). However, while robot control task is being performed, the camera data receiving should still be working.

Is there a solution for such type of programming in C++/C#?? I'm learning TBB, is it the right choice? However, I'm reading things like "loop parallelization", am I going in the right direction??

This links to a very common style in control programming where the computer is used as a central unit to connect to electronic devices (sensors) & actuators and all these devices are processed concurrently

PS: . I'm using a core i7 desktop PC, win7. I'm trying to get feedback info from a camera and control the robot with this PC, VS2008 & OpenCV 2.1

[RafSchietekat]

TBB will give you efficient use of the existing computing resources if you can expose enough parallelism in your program, but it may not give you reliable real-time control. What percentage of computing power will be needed for the tasks at hand, and would you expect to exhaust it at any time? If so, selectively degrading performance would be a challenge, I think, unlike when you bind a programming context, if I can call it that, directly to an O.S. thread that you can configure with class SCHED_FIFO or so for control and default SCHED_OTHER for vision processing. In TBB, a programming context will fan out unpredictably to run on several O.S. threads, each of them serving several contexts at different times, and there's no preemption.

[study24816]

Thank you a lot for answering! However, I'm quite a beginner. Could you make it a bit simpler?
So, you mean that TBB is not really suitable?
Then, Which tool may I use to do this? Which books can I find some directions?
and what is the name of this type of programming? is it "real-time programming"??

[Guillaume_L_B_]

TBB is mainly designed to scale while running in multi-core processor
while having lot of Chunks of data to process in parallel.

In your case, you want to run only two different tasks in parallel and I don't think you will be able to find
enough parallelizable chunk of code to give to the TBB Task Manager.
I think you can just do the jobs using two different threads and a thread safe object to exchange data between the two running threads.

However, you talked about image processing you could find some parallelization and could use TBB to process the image faster.

Kissy

[jimdempseyatthecove]

Take a look at the parallel pipeline example

Jim Dempsey

[SergeyKostrov]

Quoting study24816

...First task is, for example, get vision data from a camera & process it.
Second task is, receive processed data from first task & do sth else with this data (robot control strategy). However, while robot control task is being performed, the camera data receiving should still be working.
...

In case of a Windows platform and atwo-thread application, that uses TBBand DirectShow APIs, it should work.

[study24816]

Thank you! I've got some ideas to continue the work.
I've also reviewed the TBB book. TBB is also used for game programming, dividing rendering, physics task modules and parallelizing them. Then, I guess this applies for my problem, too.
It's more about how to break up the big task to small parallel task units.
Yet, this takes a long... time to learn. :|

[RafSchietekat]

One of the principles of TBB is optional parallelism without required concurrency, but this may not be generalisable to real-time behaviour. If vision and control are separate application/master threads, TBB will run them in separate "arenas" with additional worker threads assigned to each as needed, and no real-time issues should occur as long as control can live within a single thread and the program has at least two cores available to it. If somebody could briefly describe how this might be too restrictive, and how games deal with this (occasional glitches on the screen, however annoying, being arguably less serious than actuator errors in the physical world), that would be quite interesting (at least to me).

[SergeyKostrov]

Quoting study24816

...get vision data from a camera & process it...

What API are you going to use to get frames from the camera?

[study24816]

sorry for a short delay! as I've said, I use OpenCV and test with an ordinary webcam now. Now I'm looking for materials to practise with these thread programming.
The book I am having is "Intel Threading Building Blocks" - James Reinders. But I barely see its connection with my current task.
The book is more about parallelize data processing to threads.But I want to do it with tasks (image processing, robot arm control,...):|

[RafSchietekat]

"The book is more about parallelize data processing to threads."
That was not my impression, but I don't have the book with me to see what might make you write that?

[study24816]

This is how parallel_for, a primitive structure to implement parallelism, is used in TBB:
Foo is the function applied to each element of an array of data:

** So, a class template of this Foo is written before being fed to parallel_for:

#include "tbb/blocked_range.h"

class ApplyFoo {

float *const my_a;

public:

void operator()( const blocked_range& r ) const {

float *a = my_a;

for( size_t i=r.begin(); i!=r.end(); ++i )

Foo(a);

}

ApplyFoo( float a[] ) :

my_a(a)

{}

};

** Then, run parallel_for:

#include "tbb/parallel_for.h"

void ParallelApplyFoo( float a[], size_t n ) {

parallel_for(blocked_range(0,n,YouPickAGrainSize), ApplyFoo(a) );

}

and each process on each element are separated to different threads.

This is how I understand about TBB & its parallel structure.

But, to link this concept with what I now want to do is not easy

[RafSchietekat]

It gets less verbose with lambdas and grainsize made optional by auto_partitioner. The chunks (arguments to invocations of operator()) are distributed over the master and worker threads, but each thread processes several of those chunks, and it's always by way of tasks (I haven't kept track of whether tasks are reused across chunks), in a hierarchical architecture (your choice to tap directly into the tasks layer or not).

Maybe you'll find more information in the online documentation, as the book may not have kept track of all recent developments, e.g., separation of worker threads into arenas. Then again, the online documentation may not yet contain all information expressed in developer blogs...

What are your immediate concerns?

[SergeyKostrov]

Quoting study24816

...But, to link this concept with what I now want to do is not easy...

In one of your initial posts you've stated that you needdo getdata from a camera and then process it.
This is a two-thread or two-task concept and I think you should look at a 'tbb_thread' class instead because
it will allow to create two processingthreads.If some already received data are large then in the second
thread youcould use a 'parallel_for' to speed up processing. Remember, that if your frames are
640x480, or something like this, it doesn't make sence to processdata in parallel. Does it look good for you?

[study24816]

Ya.. I'm still working on this.. gradually. I shared my original objective in the first post. But I think it's a long way 'till I can really programme like that.
So, at first, I look for a direction, some materials that lead me to the original objective (I will firstly try with the blogs as you said)
Because, it's sad that around me, no people programme like that, I have no material, no direction to go. This forum and the experiencedforumersare my goodstart points

[study24816]

I've had a look on the blog and could only find one post quite relevant "A feature-detection example using the Intel Threading Building Blocks flow graph"

In addition... if you have known similar blogs, about computer vision programmed in TBB, or control. Please recommend me

Thanks again!

[jimdempseyatthecove]

Study24816,

If you find incorporating the operator() into your code as too obtrusive, then consider using QuickThread (or maybe CilkPlus).

QuickThread parallel_for can callstatic functions

void doWork(int iBegin, int iEnd, float a[])
{
...
}
...
parallel_for(doWork, 0, N, Array);

Member functions

struct something
{
...
void doWork(int iBegin, int iEnd, float a[])
{
...
}

};

something Obj;
...
parallel_for( &Obj, something::doWork, 0, N, Array);

And there is Lambda support as well

parallel_for( 0, N, [&](int iBegin, int iEnd){...});

Jim Dempsey

[SergeyKostrov]

Quoting study24816

Ya.. I'm still working on this.. gradually. I shared my original objective in the first post. But I think it's a long way 'till I can really programme like that.
So, at first, I look for a direction, some materials that lead me to the original objective (I will firstly try with the blogs as you said)
Because, it's sad that around me, no people programme like that, I have no material, no direction to go. This forum and the experiencedforumersare my goodstart points

Are you interested to see ashort description andsome screenshots of a real application that uses athree-thread concept?

[study24816]

Absolutely, Mr Kostrov, that will help I guess, thank you before hand!