map reduce and tbb

[norman_rubin]

I'm starting a project that will look at implementing map reduce sort of problems using shared memory and tbb
Does anyone know if there are people already working in that area?

map reduce looks like a simple combination of
a filter, a sort, and a reduction but it might need some magic to get it to be efficient

Norm

[robert-reed]

Quoting - norman.rubin @ amd.com

I'm starting a project that will look at implementing map reduce sort of problems using shared memory and tbb
Does anyone know if there are people already working in that area?

map reduce looks like a simple combination of
a filter, a sort, and a reduction but it might need some magic to get it to be efficient

Looks like no one is working on it, or at least, no one who's willing to disclose any information ;-). I hope if you make progress applying TBB to an implementation of MapReduce, you'll share your findings with our community?

[robert_jay_gould]

Norman that'd be really interesting! Good Luck!

Now assuming your success, I'd recommend you abstract away the shared-memory into some generic MapSpace container, and let users extend your MapReduce to work on on flat files, databases, or curl requests (if any of those data sources suits their needs better).

[knmaheshy2k]

Quoting - Robert Reed (Intel)

Looks like no one is working on it, or at least, no one who's willing to disclose any information ;-). I hope if you make progress applying TBB to an implementation of MapReduce, you'll share your findings with our community?

I've implemented map-reduce algorithm using systemc library (its a multi-thread C++ modeling library, but threads are co-operative). If you really want it, i can port it on to tbb and share it. We can discuss optimizations via communicator and implement.

(Added) I replied to an old post. I didn't see the post date. Sorry!

[vu64]

Quoting - knmaheshy2k

I've implemented map-reduce algorithm using systemc library (its a multi-thread C++ modeling library, but threads are co-operative). If you really want it, i can port it on to tbb and share it. We can discuss optimizations via communicator and implement.

(Added) I replied to an old post. I didn't see the post date. Sorry!

Start a new thread and share your code. Your contribution is always welcomed.