Record and playback thread interactions

[davidnoz]

I'm a PhD developer with a mathematical optimization software company.

I have developed (as a side project) some portable C code that wraps the pthreads and Windows threading API providing support for thread creation and management, mutexes, events and semaphores. The unique thing about the library is that it can record the interaction of the threads with the mutexes, events etc during a run of an application and then it can playback the interactions in a subsequent run such that the subsequent run behaves the same as the first (assuming all external input into the application runs are the same).

Using this library in my work I have not needed tools such as Thread Checker to debug my code since I have been able to detect race conditions by writing randomized test beds for my code modules and running the tests through repeated record and playback run pairs (the race conditions typically show up as deviations in the execution path that are detected by the playback i.e., thread x tries to lock mutex y instead of the expected mutex z). Furthermore, it seems that the record/playback functionality is uniquely useful when a bug occurs not as the result of a race but as the result of a particular state the application arrives at resulting from the random ordering of thread interactions with shared data.

The code is ported to 32 and 64 bit on HPUX, AIX, Linux, Solaris and Windows.

How useful is such a library?

[TimP]

You state that you have not needed tools such as Thread Checker, which take a higher level approach to diagnosing threading correctness. With either approach, completeness depends on a suite of test cases which cover all necessary code paths. With your method, it appears that you must test each build on each target with your test suite, and could (perhaps intentionally?) have technical violations of threading correctness which don't show up on your targets, with your compilation. Thread Checker aims to diagnose threading problems independent of target or compiler optimizations.

[davidnoz]

Thanks for your reply!

Its a good point that either approach depends on a suite of test cases which cover all necessary code paths. Thread Checker is perhaps more efficient in that it can detect race conditions directly whereas record/playback testing can only detect when a race causes a change in path.

With regards to the rest of your post, the library provides a set of synchronizationprimitives wrapping the system threading API (theyre based more on the Windows primitives than on the pthreads primitives i.e., recursive mutexes, events, semaphores and thread waiting). I have been careful writing the wrappers such that they behave the same on POSIX systems and on Windows. In fact (in a self referential way) I have tested the library itself across the various platforms using record/playback on randomized code path test suites and debugged it using its own record/playback functionality.

I guess in this way the approach Ive taken is also independent of target since if I can assume that the primitives behave the same on all platforms then I can develop and test threading software on Windows with a high degree of confidence that the code will work on other platforms (which I find to be the case in practice).

As for the independence of compiler optimization then I guess all I can say is that the record/playback works for all combinations of optimized/non-optimized and record/playback.