Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.

map reduce and tbb

norman_rubin
Beginner
532 Views

I'm starting a project that will look at implementing map reduce sort of problems using shared memory and tbb
Does anyone know if there are people already working in that area?

map reduce looks like a simple combination of
a filter, a sort, and a reduction but it might need some magic to get it to be efficient

Norm

0 Kudos
4 Replies
robert-reed
Valued Contributor II
532 Views
I'm starting a project that will look at implementing map reduce sort of problems using shared memory and tbb
Does anyone know if there are people already working in that area?

map reduce looks like a simple combination of
a filter, a sort, and a reduction but it might need some magic to get it to be efficient

Looks like no one is working on it, or at least, no one who's willing to disclose any information ;-). I hope if you make progress applying TBB to an implementation of MapReduce, you'll share your findings with our community?
0 Kudos
robert_jay_gould
Beginner
532 Views
Norman that'd be really interesting! Good Luck!

Now assuming your success, I'd recommend you abstract away the shared-memory into some generic MapSpace container, and let users extend your MapReduce to work on on flat files, databases, or curl requests (if any of those data sources suits their needs better).

0 Kudos
knmaheshy2k
Beginner
532 Views

Looks like no one is working on it, or at least, no one who's willing to disclose any information ;-). I hope if you make progress applying TBB to an implementation of MapReduce, you'll share your findings with our community?

I've implemented map-reduce algorithm using systemc library (its a multi-thread C++ modeling library, but threads are co-operative). If you really want it, i can port it on to tbb and share it. We can discuss optimizations via communicator and implement.

(Added) I replied to an old post. I didn't see the post date. Sorry!
0 Kudos
vu64
Beginner
532 Views
Quoting - knmaheshy2k

I've implemented map-reduce algorithm using systemc library (its a multi-thread C++ modeling library, but threads are co-operative). If you really want it, i can port it on to tbb and share it. We can discuss optimizations via communicator and implement.

(Added) I replied to an old post. I didn't see the post date. Sorry!

Start a new thread and share your code. Your contribution is always welcomed.
0 Kudos
Reply