- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Dear all, I'm always learning TBB, always with a very simple project:
- read a text file line by line
- tokenize each line and put tokens into a container
Right now I've implemented this with a pipeline, where read is a serial filter, while the tokenizer is a parallel one that puts tokens into a parallel_vector. Now I'd like to provide a "parallel tokenizer" that can work with other read/write facilities, not in an explicit pipeline. For instance, use this parallel tokenizer in another software without the user writing his pipeline explicitly.
I am open to suggestions! Is it possible to make a sort of "task" that handles tokenization?
Here's a crazy (failing) idea: a singleton with a queue, and a pipeline with a container started by the constructor, with one parallel filter (essentially the same implementation):
- another piece of code calls put(string) of the singleton
- the put(string) simply pushes the string in a parallel_queue
- the parallel filter retrieves strings from the queue, tokenize it and put it into the container
However, this crazy idea won't work. For one, how can I stop the pipeline? I could use another function that stops the flow_control, but I'm not sure. In essence, I believe this could be done with tasks, but the example for tasks (Fibonacci) isnt' really fitting.
Thanks for any hints you can give me!
Cheers!
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Update: I think it's impossible to do this with a pipeline. I tried but I cannot make it stop at will.
So, do you have any idea how I can make this work? Basically it's something like this:
class magic { public: magic() { // magically start the compute function in parallel } void insert(int i) { // this function is used by a user to insert values in a queue queue.push(i); } void stop() { // magically stop the computation } private: void compute() { // this function is magically called in parallel batches int i; // this is safe with concurrent tasks accessing the queue queue.pop(i); // do some work with the integer } tbb::concurrent_bounded_queue<int> queue; }
Thanks for any suggestion!
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
First observation: reading a file may block, but TBB is not aware of that, and so a hardware thread may largely go to waste.
To reuse code, keep it simple: start with a tokenizer that only knows about, e.g., an input string and a vector of output strings, and the user of the tokenizer should then do whatever is necessary to connect things. Don't complicate unless the performance gain to the program as a whole is significant.
![](/skins/images/7FC17B7B85029576C25F1E43CE255B51/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page