This question was asked during the webcast, An Introduction to High Performance Computing: Parallel Computing Issues. The presenter, Tom Lehmann, is the Advanced Projects Manager for the Enterprise Systems Group training organization at Intel. Here is Tom's answer to the question.
In most cases, if there is any load balancing to be done it's done by the application itself. So if I have an application that is like namd, it will do the load balancing for me. I don't have to worry about doing anything special. If I have an application that I am writing, obviously I can put as much in the way of load balancing as I want into it. If I have an application that's a parameter sweep type application, where I basically launch the same application on, say, a dozen different nodes with a dozen different data sets, it would be up to me, the person running it, to make sure that the data sets were similar enough so that I had some sort of load balancing. But I haven't seen too many applications, or I should say utilities, for doing load balancing except for, you know, estimating the amount of time a given application is going to run on a given node, and then I just manually say, "OK, I can see how I can do this."
