Intel® Moderncode for Parallel Architectures
Support for developing parallel programming applications on Intel® Architecture.
1690 Discussions

Debugging thread parallel applications on Intel platforms

Hi, I'm Gordon, the Debugger Consultant for Intel's Software Products Division. I'm responsible for enabling and promoting effective debugging on Intel platforms in general, and with the Intel Debugger in particular.
My current focus is on creating a repository of Best Known Methods for debugging serial, thread parallel and cluster parallel applications.
I'd like to get a feel for how the community at large approaches application debugging. I'm curious to know things like:
  • What types of bugs do you encounter most often?
  • What types of bugs do you find most difficult to locate and fix?
  • What methods do you use to accomplish this?
  • Which debuggers do you tend to use (and why)?
  • Which debuggers do you prefer to use (and why)?
  • Which debugger features do you find yourself using most often?
  • Which new debugger features would you consider to be essential?
Please feel free to contribute accordingly. I appreciate your participation, it helps significantly to improve both the processes and the tools for all of us.
0 Kudos
2 Replies

Hi Gordon,

You're not likely to get any best known debugging methods from me. I still use the good ol' print statement quite a bit. I'll start the discussion anyway.

I can't really single out any particular class of bug that I encounter most often. I mainly work on parallel codes so storage conflicts and deadlock are common problems. When debugging threaded codes, I use the Intel ThreadingChecker most often. I've been using theIntel Thread Checker for a long time. Itcan find some pretty heinous bugs faster and with a lot less effort than a traditional debugger. For MPI codes, I use TotalView (Etnus). Why do I use TotalView instead of the Intel debugger? Because I've been using TotalView foryears and I'm comfortable with it. I also like the GUI.



Thanks Henry. No apologies necessary for printf. Its useful. In fact, IDB gives you the ability to instrument your code with printf without your having to modify and rebuild your code to make the insertions (and then again to remove them).

You can accomplish that by using IDB statements as simple as

when at 200 { printf "%d ", i }

Or you could make use of the many related qualifiers to do something much more specific like

when quiet at 200 thread 1 if x < 10 { printf "Value of x is 0x%x, of i is %d ", x, i }

In addition, you can put these commands in a .dbxinit initialization file in the directory from which you run the program, which makes them automatically available to you each time you invoke the debugger on that program (from that directory). Turning them off is a simple matter of saying disable all. Turning them back on again just requires enable all.

You may also want to take a look at the internal Debugging Tips and Techniques Newsletter for more helpful hints. Issues of the newsletter will be made available here in this forum starting next month.

You mentioned the Intel Thread Checker (ITC) which is an excellent debugger for threaded codes. Have you considered a multi-step approach to debugging such as:

  1. Test new (and newly modified) code without threads enabled. Debug using IDB.
  2. Test with threads enabled and limited to a single thread. Debug with ITC (and then - if necessary - with IDB).
  3. Test with more than one thread. Debug with ITC (and then - if necessary - with IDB).

Its possible that general logic problems exist but that dont manifest themselves until exposed to a more challenging environment (which is where IDB comes in during the later steps).

You also mentioned Etnus TotalView (which I believe to be the Gold Standard in debugger technology in general) for debugging cluster parallel applications. There are actually several good tools available for debugging clustered codes. For instance, theIntel Cluster Tools (i.e. Intel Trace Collector and Intel Trace Analyzer) can be used together as an effective debugger on clustered codes. Both IDB and Streamline DDT are also viable debugger options. IDB is provided without cost on the Intel Compiler kits and provides a nice clean command line interface (with an aggregation network for concentrating program and debugger output) for clustered codes. Streamline DDT is a premium debugger that provides more graphical features while coming in at a lower cost per seat than Etnus TotalView.

The Trace tools and one of the debuggers could be used together in a multi-step process (like the one described above for threaded codes) to achieve a more efficient debugging experience.

What about other commun ity members, do you have additional thoughts in this regard?