Learning to parallelize code

[David_DiLaura1]

Steve (or anyone):

I've been writing engineering analysis code in Fortran for many years and it's time to bring parallelization to what I do. I have read a few things (like the document "Parallel Programming in Fortran 95" by Hermanns) but the devil (and success!) is in the details. The systems I work on typically have units with ~10,000 lines of code and large arrays. So naturally my initial and newbie attempts result in Stack overflows . . . but even allocating max stack space still produceserrors when OpenMP attempts to establish multiple threads ("insufficient memory to perform this function"). Rather than thrash around, I thought there might be more and better information that is available and that would help me by reading it first.

David

[Steve_Nuchia]

Always start by inderstanding where your code is spending its time. Use a good clean sampling profiler, VTune is the best I've seen in Windows, and once you've identified the hotspots it will be a lot clearer what you need to do next.

There is fundamental decision to be made as to the level at which you want to introduce parallelism. If you were starting from scratch it might make sense to organize your entire program around a work item queue or something like that. At the other extreme you can easily convert a typical linear-algebra-heavy program to call a pre-parallelized math library. Generic advise if pretty much worthless I'm afraid.