topic Re: Why Pthreads are better than Win32 threads in Intel® Moderncode for Parallel Architectures

Why Pthreads are better than Win32 threads

ClayB — Wed, 14 May 2003 00:28:46 GMT

I've used both POSIX threads (Pthreads) and Win32 threads APIs and I believe that Pthreads has the better programming model of the two. While each threading method can create threads, destroy threads, and coordinate interactions between threads, the reason I make this claim is the simplicity of use and elegance of design of Pthreads. Let me illustrate with a few examples.

Separate data types. In Pthreads, each object has its own data type while in Win32 threads there is a mix of handles and separate types. For Pthreads this means different functions are used for working with each object type. Reading and understanding Pthreads code written by someone else is straightforward and less apt to lead to confusion. On the other hand, because of the use of the same type for different objects, when a Win32 program uses WaitForSingleObject, it may not be readily apparent if the code is expecting a thread termination, an event to be signaled, or a mutex to be released. This also illustrates my next point.

Unambiguous functionality. I've actually seen Win32 code that used an array to hold both thread and mutex handles, then wait on those handles and execute different code paths depending upon which handle was signaled first. Correct implementation, yes, but a tough nut to understand on a first reading. While Pthreads may have more functions defined (around 60) than Win32 threads (I counted close to 30 thumbing through a book on Win32 threads programming) Pthreads has a single function to create threads. If you include the C Runtime Library, there are three separate ways to do this for Win32 threads.

Persistence of signals. It is up to the programmer to ensure the proper switching of Win32 events from the signaled to unsignaled state. Part of this involves setting the attributes of the event correctly and resetting manual events. If either of these is not done (and I've been guilty of not properly initializing or resetting events many times), the application will not function as expected. Tracking down this error can be difficult even with debugging tools. Under Pthreads, signals to condition variables are either "caught" by waiting thread(s) or discarded. However, use of a well known coding structure at each access of a condition variable will ensure no signals are "lost" by threads that may not be waiting at the exact time of signaling.

These are just some of the reasons that I think Pthreads is a better threading API than Win32 threads. What do you think? Do you have other reasons to prefer Pthreads? Or do you think Win32 threads is the better threading method? I'd like to hear about your preferences.

Re: Why Pthreads is better than Win32 threads

robert-reed — Wed, 14 May 2003 00:56:18 GMT

I know at one time that Pthreads were treated as second class citizens in the Windows environment. Has Services for Unix resolved that prejudice? Or are there still disparities in the way the kernel these two APIs?

Re: Why Pthreads are better than Win32 threads

rdwells — Wed, 14 May 2003 01:53:36 GMT

You missed one other potential advantage: portability. The pthreads interface is native for Mac OS X and Unix variants, and has been implemented for Win32 via an open source project (I don't have the URL off the top of my head, but I can look it up if anyone wants).

Note that I called it a "potential" advantage. I haven't worked with the open source implementation for Win32 mentioned above, so I don't know how well it works. But if it does work, I can have my Windows and Mac threading support shared between the two platforms. (How ironic that I would do this via a Unix-defined interface!)

Re: Why Pthreads are better than Win32 threads

randy_howard — Wed, 14 May 2003 05:23:55 GMT

The portability issue is the main one, I agree. The pthreads-win32 implementation (available from RedHat ironically) is available under LGPL licensing for Windows development. Not only does it work, but it is quite stable and in performance comparisons with real-world code implemented both ways, it is difficult to measure any difference in performance. The advantage of being able to use pthreads in a lot of different programming environments is an obvious win, without giving up anything on Windows.

Further, you can use pthreads in some surprising places now, such as Novell NetWare NLMs (using libc instead of clib for NLM modules). I have several production pthread applications that run on multiple Linux, Windows and Novell platforms from a common code base with some very minor #ifdef FOO_PLATFORM stuff added in, none of which is related to pthreads differences btw.

Last I heard, the Mac implementation isn't 100% baked yet, even under Jaguar, but I haven't tried it yet personally.

Re: Why Pthreads are better than Win32 threads

ClayB — Wed, 14 May 2003 06:16:30 GMT

Thank you rdwells and Randy!

I did a Google search on "pthreads win32" and found the Red Hat location for this. Unfortunately the site wouldn't come up when I tried it, but the third link in the Google list sent me to a Hong Kong mirror site that was accessible.

It looks to be a simple DLL that you can link in with your Pthreads source code. The README notes that you need to choose only one from several variations of the binary or you can take the source and compile it yourself.

If and when someone tries this library, please go ahead and start a new topic for discussing the use of the DLL as well as any tricks and traps that users might run into.

-- clay

Re: Why Pthreads are better than Win32 threads

TimP — Wed, 14 May 2003 21:07:03 GMT

I suppose that the pthreads library of cygwin is similar. The gcc testsuite threaded tests work fine on a P4 HT platform, as long as the test suite itself is run serial. I haven't looked at the failures, when the test suite is run with make -j 2, to see whether there is a problem running multiple jobs, each of which uses Windows pthreads.

Re: Why Pthreads are better than Win32 threads

Henry_G_Intel — Thu, 22 May 2003 05:35:06 GMT

I'm surprised no one took the bait for Clay's deliberately provocative subject heading, especially since the Windows threading API is infinitely better than Pthreads. For example, Win32 mutexes are safer than Pthreads mutexes. If a POSIX thread tries to lock a mutex that it already owns, self-deadlock is possible. If a POSIX thread terminates while holding a mutex, subsequent threads attempting to acquire this mutex are deadlocked. What happens if a POSIX thread tries to lock an uninitialized mutex or unlock a mutex that it doesn't own? A good Pthreads implementation will detect these programmer errors but it's not required for POSIX compliance. The behavior of the Windows threading API is clearly defined for each of these usage errors.

Re: Why Pthreads are better than Win32 threads

Intel_C_Intel — Sun, 25 May 2003 04:46:51 GMT

> I'm surprised no one took the bait for Clay's
> deliberately provocative subject heading, especially
> since the Windows threading API is infinitely better
> than Pthreads.

Almost the entire Windows threading is hopelessly BROKEN.

Most Win "applications" are full of race conditions and utter silliness ala dying "spinlocks", and etc.

http://sources.redhat.com/ml/pthreads-win32/2001/msg00158.html

> For example, Win32 mutexes are safer
> than Pthreads mutexes. If a POSIX thread tries to
> lock a mutex that it already owns, self-deadlock is
> possible. If a POSIX thread terminates while holding
> a mutex, subsequent threads attempting to acquire
> this mutex are deadlocked.

http://groups.google.com/groups?threadm=3DE17AC9.4010405%40arcor.de

> What happens if a POSIX
> thread tries to lock an uninitialized mutex or unlock
> a mutex that it doesn't own? A good Pthreads
> implementation will detect these programmer errors
> but it's not required for POSIX compliance. The
> behavior of the Windows threading API is clearly
> defined for each of these usage errors.

Well, you should take a Java^H^H^H^H C# class, I guess. ;-)

regards,
alexander.

Re: Why Pthreads are better than Win32 threads

ClayB — Wed, 28 May 2003 06:31:26 GMT

I was, of course, hoping to start some controversy and generate some discussion on the merits of both Pthreads and Win32 threads (assuming that proponents of both were involved with the Threading Forum). Alas, almost everyone that has taken the time to voice an opinion has agreed with me. Not much chance of controversy there.

I've looked at the poll on what threading API is used most by respondents and found almost twice as many Win32 users as Pthreads users. Are there no Win32 advocates or do they agree with what the Pthreads supporters have said, but use the API for reasons beyond their control?

I have no answers and I really didn't want this to degenerate into a bashing of either API. Perhaps we can just let this topic fade away quietly?

-- clay

Re: Why Pthreads are better than Win32 threads

jseigh2 — Wed, 28 May 2003 19:04:02 GMT

Part of the problem is that you don't really have a choice what platform you use. To say that you have a choice on win32 because you can use the pthread-win32 (or whatever they call it) package. For one reason, see this article. For another, there's too much of a runtime difference between unix and windows. You'd have to port the entire unix runtime to windows. That's already been done. It's called linux.

If all you want is a condition variable, then you should just implement a condition variable without all the posix stuff. BTW, I did implement a win32 condvar without the problems mentioned in the article, so it can be done. Even if you don't read the article, you can take it that it's non-trivial enough that Microsoft put condvars in C#, otherwise there would have been no way to port Java apps to C# using win32 events and still have those apps work. Non-trivial enough that it was an interesting challenge for me to do it in C.

There is, however, some things that win32 threads did better then posix threads. The WaitForMultipleObjects allows you to decouple the signaling and waiting threads better. In posix, if some thread wants to wait on multiple conditions, there has to be a condvar especially for that and the signaling thread has to know about it. In win32, the signaling thread only has to signal the events it knows about and not have to worry about other threads waiting for mulitple events, though there still the problem of events being lossey. Also win32 threads are more orthagonal. You do not have have totally different and incompatible signaling mechanisms like unix select/poll and pthread condvars. The kludges to make those work together are incredible.

Joe Seigh

ps. note to Intel. I you want more participation in these forums, you are going to have to make this website more browser friendly. It is a major undertaking to post anything here.

Re: Why Pthreads are better than Win32 threads

Intel_C_Intel — Wed, 28 May 2003 21:48:12 GMT

> Part of the problem is that you don't really have a
> choice what platform you use. To say that you have a
> choice on win32 because you can use the pthread-win32
> (or whatever they call it) package. For one reason,
> see
>
> his article.

Read also this:

http://google.com/groups?selm=3AEAC433.1595FF4%40web.de
(Subject: Re: A theoretical question on synchronization)

> For another, there's too much of
> a runtime difference between unix and windows. You'd
> have to port the entire unix runtime to windows.
> That's already been done. It's called linux.

It's called Interix/(POSIX) Windows sub-system, Joe.

http://google.com/groups?selm=3DE4FF8C.FA94CDAC%40web.de

> If all you want is a condition variable, then you
> should just implement a condition variable without
> all the posix stuff. BTW, I did implement a win32
> condvar without the problems mentioned in the
> article, so it can be done.

Joe, please, don't get me started on your CVs with "non-blocking" DCCI. Really. Please. ;-)

> Even if you don't read
> the article, you can take it that it's non-trivial
> enough that Microsoft put condvars in C#, otherwise
> there would have been no way to port Java apps to C#
> using win32 events and still have those apps work.
> Non-trivial enough that it was an interesting
> challenge for me to do it in C.
>
> There is, however, some things that win32 threads did
> better then posix threads. The
> WaitForMultipleObjects allows you to decouple the
> signaling and waiting threads better. ...

WHAT?!

http://groups.yahoo.com/group/Boost-Users/message/3904
(Subject: Re: combinations of boost::condition)

regards,
alexander.

Re: Why Pthreads are better than Win32 threads

ClayB — Wed, 28 May 2003 21:58:46 GMT

Joe, thanks for posting the article pointer. Even if you aren't interested in the creation of POSIX condition variables with Win32 constructs, it is informative about the problems and subtleties of programming with threads. While a simple and easy example to understand, it is unfortunate (to some degree) that the authors chose to implement a semaphore object with the POSIX condvar when Win32 already supports such a thing (as well as POSIX, I believe).

Also, while I'm on my soapbox, there are two reasons for putting the pthread_cond_wait into a while loop. The first is as the article mentions, if a thread waiting on the condition variable is signaled, it must first recheck the condition being waited on before proceeding. In order to be safe and catch all possible situations, every time some operation is performed that may change the evaluation of the condition for a waiting thread, a signal should be generated. The thread that wakes up reacquires the lock, checks the condition in the while test and either proceeds or finds the condition lacking and goes back to the wait state.

The second reason for putting a while test around the pthread_cond_wait is spurious wakeups. It is rare, but still possible that a thread will return from waiting without a signal or broadcast being generated by another thread on the condition variable. Since no thread has signaled, it is unlikely that the condition being waited on has been satisfied, so the while test forces the thread back into the wait state.

Having learned and used Pthreads before Win32 threads, I was initially dismayed that there was no condition variable (or something like it) in the Win32 threads API. But, it's like using two different programming languages, say Fortran and C++. They both have many things in common that may be expressed in slightly different syntax, and there are features in each that aren't in the other. This just means that you need to modify your approach when porting from one language (threading API) to the other. I mean, would anyone try to implement Fortran array syntax in C++ or objects in Fortran? Some might, but the majority would just work within the confines of what is available.

Have you seen or tried the Red Hat implementation of Pthreads on Windows? There is a link in an earlier post. While the original intent was not to judge the two different threading APIs on performance (since, as was pointed out, they are implemented on two different runtime systems), but now we have an implementation of both on the same OS.

-- clay

Re: Why Pthreads are better than Win32 threads

Intel_C_Intel — Wed, 28 May 2003 22:10:01 GMT

> Joe,
[...]
> Have you seen or tried the Red Hat implementation of
> Pthreads on Windows?

He has NEVER heard of it, believe me. ;-) ;-)

regards,
alexander.

Re: Why Pthreads are better than Win32 threads

Henry_G_Intel — Fri, 06 Jun 2003 23:42:01 GMT

Hi Clay,
I downloaded the Red Hat Pthreads implementation and built the DLL on my system. I linked it to a Pthreads program on Windows and it ran correctly. It's a pretty easy way to get Pthreads programs working on Windows. I didn't do any performance experiments but I don't expect a significant performance loss. The Red Hat Pthreads implementation just wraps equivalent Windows threading functions.

I did an experiment to test whether the Intel Thread Checker could analyze a Pthreads program on Windows. The Pthreads DLL was built with the /Qtcheck option in the Intel compiler. My test program was also built with the /Qtcheck option. Data races were deliberately introduced into the program. Thread Checker correctly identified the data races, their locations in the code, and the variables involved.

Henry

Re: Why Pthreads are better than Win32 threads

davids — Wed, 11 Jun 2003 14:24:27 GMT

I really don't like either. I generally use a set of C++ classes that I implemented both for pthreads and for WIN32 threads. I get the advantages of complete portability and an API that I consider perfect. I have classes like 'Mutex', 'SpinLock', 'Event', 'ThreadPool', 'Timer', and so on.

Re: Why Pthreads are better than Win32 threads

Henry_G_Intel — Wed, 11 Jun 2003 20:59:01 GMT

I agree that abstracting native thread libraries in classes is a good thing, but if you have to move a C, Pthreads code to Windows, the Red Hat Pthreads DLL makes porting a lot easier.

Re: Why Pthreads are better than Win32 threads

randy_howard — Fri, 11 Jul 2003 19:12:05 GMT

> I agree that abstracting native thread libraries in
> classes is a good thing, but if you have to move a C,
> Pthreads code to Windows, the Red Hat Pthreads DLL
> makes porting a lot easier.

A futher reason is if you happen to be so lucky (???) as I in that you need to develop code that will run on Windows, Linux and Novell (gasp) platforms from a common code base. libc NLM's now have pthreads available, particularly when running with the latest service packs applied. I've successfully migrated several pthreads applications from Win32/Linux to Novell with only minor code changes.

I have reported a few issues to them that are being investigated, but they are quite minor.

Short version: If you want to develop for multiple platforms, including Windows, Linux, Novell, MacOS and a few others, pthreads is the only way to go. If you're happy working on only WIN32 for the rest of your career, then put up with that abomination. :-)

I agree with whoever it was that recently said this is a horrendous UI for posting messages. It reminds me of what you would have done in the FidoNet BBS days if GUI's had been written by the same developers that implemented those bulletin board systems.

Re: Why Pthreads are better than Win32 threads

rosspjohnson — Fri, 22 Aug 2003 00:39:58 GMT

> I downloaded the Red Hat Pthreads implementation and
> built the DLL on my system.

Just a minor clarification here - Pthreads-win32 is generously hosted by Red Hat but they don't implement it. Historically, the initiator of the project worked for Cygnus Solutions, which then became part of Red Hat. It has always been an independently produced library, with many significant contributors.

Indeed, the principal architect of the CV code in pthreads-win32 was Alexander Terekhov (previous messages in this discussion), and it has proven to be extremely robust.

Ross Johnson
Pthreads-win32 maintainer

Re: Why Pthreads are better than Win32 threads

Henry_G_Intel — Sat, 23 Aug 2003 01:39:26 GMT

Hi Ross,
Thanks for clarifying. I agree that this is a robust, easy-to-use package. It's very modest of Alexander not to take credit.

Henry

Re: Why Pthreads are better than Win32 threads

battlecattle — Mon, 26 Feb 2007 23:06:56 GMT

I guess the author has changed his mind. Here is his take on why Win32 threads are better...

http://softwareblogs.intel.com/2006/10/19/why-windows-threads-are-better-than-posix-threads/

"Ive used both POSIX threads (Pthreads) and Windows threads APIs, and I believe that Windows has the better programming model of the two. While each threading method can create threads, destroy threads, and coordinate interactions between threads, the reason I make this claim is the simplicity of use and elegance of design of the Windows threads API. This is all from the perspective of multithreaded code developers or maintainers. Let me illustrate with a few examples.

Simplicity of data types. In Pthreads, each object has its own data type (pthread_t, pthread_mutex_t, pthread_cond_t, etc.) while, in Windows threads, there is pretty much just the one type: HANDLE. For Pthreads this means different functions are used for working with each object type. Reading and understanding Pthreads code written by someone else can be straightforward. However, this does mean that the programmer must know the number, order, and type of parameters for all the different functions. On the other hand, because of the use of the same type for different objects, there is a Create* function for each different object and a corresponding Release* function for most.

Perhaps the biggest advantage of a single object data type is that there is only the one function needed to make a thread block while waiting for an object: WaitForSingleObject. Thus, only one set of parameters needs to be known regardless of whether the code is waiting on a thread, a mutex, a semaphore, or an event. The related function, WaitForMultipleObjects, is just as simple to use and easily overcomes the problem of needing to wait for multiple thread terminations one function call at a time (pthread_join) that Pthreads requires. While some may say that using a single data type for many different objects can lead to confusion when used in WaitFor* calls, programmers should set the name of the handle such that it is readily apparent whether the code is expecting a thread termination, an event to be signaled, or a mutex to be released.

WaitForMultipleObjects functionality. Besides being able to block a thread waiting for multiple thread terminations in a single call, the programmer can actually wait for any out of a set of threads to terminate. That is, even when only one thread has completed, the WaitForMultipleObjects function can be set to return and indicate which thread triggered the return. If there is specific clean up processing that depends on the identity of the thread that finished, this can be done before returning to wait on the remaining threads. This clean up processing will be done in the most efficient order possible, soon after each thread terminates, no matter in what order this happens. Pt hreads can perform similar post-processing, but will need to wait for the threads to terminate is some fixed order. So, even if the last thread finishes first, it must wait for all the post-processing of the previous threads to be completed.

Because different objects all use the HANDLE type, a call to WaitForMultipleObjects can be set to wait for any combination of threads, mutexes, semaphores, and/or events. This feature can give the programmer a flexibility that cannot be easily (if at all) duplicated in Pthreads. As an example, Ive written Windows code that used an array to hold both thread and event handles to support a threaded search through data. The ideas was to signal the blocking thread if the item being looked for was found and when the searching thread terminated; if the object was not found, the searching thread terminated without setting the event. By waiting for either handle to cause WaitForMultipleObjects to return, a simple switch statement could determine if the item had been found (and process the data) plus perform some post-processing computation upon the termination of the searching thread (regardless of whether the search was successful).

Persistence of signals. To paraphrase a classic conundrum in terms of Pthreads: If a thread signals a condition variable and no other thread is waiting, does it make a sound? The signal is lost if there is no thread waiting on the condition variable. For this reason, it is mandatory to set up a while loop to test a conditional expression (used to prompt a signal), and that requires getting a mutex involved to protect the data within the conditional expression and any changes to that data, and dont even get me started about spurious wake ups. Sheesh!

For Windows threads, once an event is in the signaled state, it stays signaled. In other words, when that tree falls, it continues to scream in pain until someone comes along to hear it. It is up to the programmer to ensure the proper switching of Windows events from the signaled to unsignaled state. Part of this involves setting the attributes of the event correctly and being sure to reset manual events as needed. All in all, this seems more convenient and simple. Besides, once the specified conditions have been achieved in order to signal the event, its not like they can be unachieved later. Under Pthreads, if you blink, youve missed it.

These are just some of the reasons that I think Windows threads is a better threading API than Pthreads. What do you think? Do you have other reasons to prefer Windows threads? Or do you think Pthreads is the better threading method? Id like to hear about your preferences.

clay"