A New Parallel Programming Book: "The Art of Multiprocessor Programming"

[AJ13]

I've been reading way too many books, but I'm ordering this one now. Looks pretty good!

http://www.amazon.com/Art-Multiprocessor-Programming-Maurice-Herlihy/dp/0123705916

AJ

[syzygies]

Yes, this is a great book, by top researchers in the theory of parallel computing. Essential theory, and a conceptual "practice" section that gives imagination as to what can go wrong. I know researchers who've been writing parallel code for years using locks, but not keeping up with the literature, that were stunned by some of the results I learned from this book.

This was the most rewarding of a handful of parallel books I bought recently. If anyone knows something better (O'Reilly Intel TBB is obviously essential reading, but less conceptual) I'm all ears.

The psychic disconnect I get is their view that C is simply not thread-safe, as if it is beyond repair, with every thread-safe language relying on virtual machines that give away C's performance. Yet here, clearly, Intel gets TBB to work with C++. Nevertheless, I see various questions here about thread-safe operations, as if the whole issue is still a black art. It is a sorry state of affairs if there is no compiled language where one doesn't have to worry about this.

(I use Haskell because it compiles faster in my brain; the cpu is there to serve me, not the other way around. And it nicely supports various parallel paradigms. But I want to try TBB on a few problems, for comparision.)

[AJ13]

I think that you will find that Haskell conceptually maps quite well to parallel programming. I tend to think in terms of functional programming myself at times, depending upon what I'm doing.

I'm hoping that this book will help with understanding lock-free algorithm design and optimimal concurrent data structure design. I ordered it, and I'm excited to see what it's like. Another book I have is patterns for parallel programming, you might like it. Check it out here: http://www.amazon.ca/Patterns-Parallel-Programming-Timothy-Mattson/dp/0321228111/ref=sr_1_1?ie=UTF8&s=books&qid=1210771327&sr=8-1

I've been reading the Multicore pack from Intel Press. It hasn't been as technical as I want it to be, but a good read.

So far as C++ supporting threading, C++0x the new standard has built in threading support. My understanding is that the C standards people are looking to C++0x for the conceptual threading model to adapt it to their needs.

[Dmitry_Vyukov]

Quoting - syzygies

The psychic disconnect I get is their view that C is simply not thread-safe, as if it is beyond repair, with every thread-safe language relying on virtual machines that give away C's performance.

Take a look at their (Maurice Herlihy, Nir Shavit) latest research - Hopscotch Hashing:

http://groups.google.com/group/hopscotch-hashing

Try to guess what language they use for implementation/benchmarking. Haskell? Nah. Java? Nah. C++? Is it possible? Yeah, plain old C++.

Don't get blinded by marketing!

[Dmitry_Vyukov]

Quoting - Dmitriy Vyukov

Take a look at their (Maurice Herlihy, Nir Shavit) latest research - Hopscotch Hashing:

http://groups.google.com/group/hopscotch-hashing

Try to guess what language they use for implementation/benchmarking. Haskell? Nah. Java? Nah. C++? Is it possible? Yeah, plain old C++.

Don't get blinded by marketing!

Just a few questions.

How are you going to prevent colocation of some objects in one cache-line (false-sharing), which can be crucial for design of synchronization primitive, in Java/Haskell?

How are you going to enforce colocation of some objects in one cache-line, which can be crucial for design of synchronization primitive, in Java/Haskell?

How are you going to express relaxed atomic RMW operations in Java/Haskell?

Maurice Herlihy and Nir Shavit use exactly such things in Hopscotch hash map. They use packing of items into single cache-line. They use extremely fine-grained atomic operations, for example load-release-wrt-other-loads (used in timestamp validation).

In C/C++ all those things are no problem. One can do anything he wants.

[Dmitry_Vyukov]

Quoting - aj.guillon@gmail.com

I think that you will find that Haskell conceptually maps quite well to parallel programming. I tend to think in terms of functional programming myself at times, depending upon what I'm doing.

Functional languages have many useful concepts, but many (all?) of them can be used in imperative languages as well.

Uniqueness types? No problem. Just implement appropriate smart pointer.

Immutable objects? No problem. Just implement appropriate wrapper.

Asynchronous message-passing? No problem.

Many such concepts I'm currently using in C++.

[dharaj]

In addition to the concept matches posted above, C++0x will have features that will make it easier to map some hallmark features of functional languages to C++, such as polymorphic functions and better support for currying. There's already a functional C++ library called FC++.

If you really like functional programming's way of thinking, I reccommend you look at that library. I too enjoy Haskell and I was glad to find out that there is a library for C++ that brings a lot of its ideas to the language and that it'll be easier to implement those ideas in the next standard.

[dhruva]

Quoting - dharaj

There's already a functional C++ library called FC++

I suggest using BOOST (http://www.boost.org). With quite a few BOOST functionalities entering standard C++, that would be the best place to start if you want powerful extensions to C++ (over STL).