Link Copied
Raf_Schietekat:
About Arch's message, paragraph by paragraph:
A full fence defined this way leaves out sequential consistency, strictly speaking (memory_model_acq_rel vs. memory_model_seq_cst), which may or may not become visible on some processors (which ones?), though not on IA-32/Intel 64. Which processors might that be, and what are the performance considerations for making this (farfetched?) difference? Dmitriy?
I'm not ready to answer about processors, but there are definitely some processors without total store order (I think that it's SPARC not in TSO mode, maybe PPC, maybe ARM).
Full fence without sequential consistency can lead to very counter-intuitive results. I think that seq_cst added mainly to provide clear and easy-to-reason semantics. And to provide tool for prototyping.
See page 11 in presentation "The Future of Concurrency in C++":
http://www.justsoftwaresolutions.co.uk/files/future_of_concurrency.pdf
You can replace release and acquire fences with acq_rel (full) fences - it will not change the output.
As you can see output is very counter-intuitive.
See page 12 where seq_cst is used. Result is intuitive.
Why regrettable, if they are hidden in locks and atomics? It seems a bigger problem that now IA-32/Intel 64 does not allow a way out to provide cheap raw/unfenced/relaxed atomic operations, especially involving locked operations, which now impose a full fence (also see below).
Agree. But I don't think that we can change anything here :)
The question is whether to allow all combinations orthogonally, or to impose any perceived wisdom on the user through undefined template specialisations. The challenge is to do this across all architectures (do they have incompatible sweet spots or can they agree?).
AFAIK, Current C++0x standard prohibits only store-acquire, store-acq-rel, load-release and load-acq-rel.
I don't know rationale behind this.
[...] fences may exist in the following combinations: [...] unfenced/reload/flush/reload+flush as another sequential consistency-related family.
Please describe semantics of unfenced/reload/flush/reload+flush in more detail.
Dmitriy V'jukov
With a bit of elementary-particle physics, fences may exist in the following combinations: unfenced/acquire/release/release+acquire as one causality-related family, and unfenced/reload/flush/reload+flush as another sequential consistency-related family. In addition to what their counterparts in the first family do, a reload will not move up across a flush, which gets rid of relativity (mixing my metaphores), but I don't know yet if that is all that is different. Then there's the matter of what it means to have a device that unidirectionally prevents reordering of anything (does it make sense at all?), the meaning of LoadStore, rationale for stand-alone fences, a lot of other things and whether they are relevant or just there to confuse me... So I'm still wondering about such matters, and Dmitriy just sabotaged my attempt to limit my scope.
For now I stick to following approach.
First of all, whenever possible I use C++0x interfaces, semantics and names. For example, unfenced/naked vs relaxed. It seems that for now C++0x is sufficiently stable wrt memory model and atomics. Current C++0x draft:
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2008/n2588.pdf
In my personal opinion, C++0x provides too restrictive semantics. Luckily it's not only my opinion, so I also take as
basis proposals for C++0x memory model/atomics from Peter Dimov, Alex Terekhov, Paul McKenney:
http://groups.google.com/group/comp.programming.threads/msg/b43cd6c9411c95b9
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2237.pdf
I hope that they think enough before making proposal, so I can not think :)
Initially I was going to implement atomics with templates:
atomic_store
I was thinking that it's the only efficient method. And following approach will invietably add unnecessary overheads:
atomic_store(&x, 1, memory_model_something);
Finally I figure out how to implement atomics with C++0x interface (atomic_store(&x, 1, memory_model_something)) in efficient and comfortable way. Here is code sketch:
/*** Pay attention to inheritance ***/
// fake root
struct memory_order {};
/***** full fences *****/
// full fence + total order
struct mo_seq_cst_t : memory_order {};
// full fence
struct mo_acq_rel_t : mo_seq_cst_t {};
/***** release fences *****/
// classic release
struct mo_release_t : mo_acq_rel_t {};
// release not affecting stores
struct mo_release_load_t : mo_release_t {};
// release not affecting loads
struct mo_release_store_t : mo_release_t {};
/***** relaxed fences *****/
// does not order memory
struct mo_relaxed_t : mo_acq_rel_t {};
extern mo_seq_cst_t memory_order_seq_cst;
extern mo_acq_rel_t memory_order_acq_rel;
extern mo_release_t& nbsp; memory_order_release;
extern mo_release_load_t memory_order_release_load;
extern mo_release_store_t memory_order_release_store;
/*** Implementation for MSVC/x86 ***/
// Only store - other operations stripped
class atomic32
{
public:
typedef unsigned value_type;
explicit atomic32(value_type v = value_type())
: value_(v)
{
}
void store(value_type v, memory_order = memory_order_seq_cst) volatile
{
_InterlockedExchange((long*)&value_, v);
}
void store(value_type v, mo_release_t) volatile
{
_ReadWriteBarrier();
value_ = v;
}
void store(value_type v, mo_relaxed_t) volatile
{
value_ = v;
}
private:
value_type volatile value_;
atomic32(atomic32 const&);
atomic32& operator = (atomic32 const&);
/*** forbidden ***/
void store(value_type v, mo_acq_rel_t) volatile;
};
Basic rule: I always provide implementation for seq_cst. Then I provide specialized implementations, if they can be implemented in more effective way on current architecture (release). Then I move forbidden combinations to private section (acq_rel).
All other things are handled by inheritance. I.e. store with mo_acquire_t is also forbidden. Store with mo_release_store_t uses implementation for store with mo_release_t.
Also I add a bunch of compiler fences (affects only code generation):
/***** compiler fences *****/
struct co_acq_rel_t : mo_acq_rel_t {};
struct co_acquire_t : co_acq_rel_t {};
struct co_release_t : co_acq_rel_t {};
I'm not sure about bidirectional fences (store-store (sfence), load-load (lfence) ). For now I just comment them out.
What do you think?
Dmitriy V'jukov
With respect to the reference-counting example, my impression is that it is typically unsafe to leave out the fence, because the increment of the reference count typically implies that a thread is acquiring rights to read a shared reference-counted object.
Provided so-called 'basic thread-safety' (thread permitted to acquire reference to object *only* if it aready has one, very common case, for example boost::shared_ptr) it's ok to remove fence, because thread doesn't actually acquiring anything.
Provided so-called 'strong thread-safety' (thread permitted to acquire reference to object even if it doesn't have one, very uncommon case) it unsafe to remove fence, because thread actually acquiring access to object.
Dmitriy V'jukov
Provided so-called 'basic thread-safety' (thread permitted to acquire reference to object *only* if it aready has one, very common case, for example boost::shared_ptr) it's ok to remove fence, because thread doesn't actually acquiring anything.
In this case mechanism which transfers object between threads (producer-consumer queue) must execute acquire fence, because this mechanism gives access to object to thread.
Dmitriy V'jukov
That p. 11 example is not to the point for illustrating acq_rel, I would say.
It illustrates difference between acq_rel and seq_cst.
As for allowing all combinations orthogonally vs. imposing the designer's wisdom I might be convinced by the argument that sometimes a language gives you more by the things it does not allow you to do (but I can't say now where I've heard that before).
I think you are right. This will eliminate at least a bit of complexity.
But I am thinking about special cases. For example sequence lock requires load with memory_model_release_wrt_loads (or msync::slb in Terekhov's list).
In the second family I mentioned, reload would mean "forget any cached reads" (not actually reload, but the best word I could find was what it would imply later on), and flush would mean "write all dirty entries to memory now and wait until finished"; I don't know how useful flush and reload might be by themselves, though.
It looks quite unusual and confusing...
Oh boy, Alexander Terekhov's list again... I tried to ignore that for lack of a legend (what do the entries all mean?)
There is a legend. Imho the semantics are clear...
, so I obviously don't know whether it's useful at all. So now it really is all back on the table. One problem is that too much detail might be too confusing and lead to bugs.
I think, that it's better to dive into as much detail as possible, and then try to climb as high as possible. It's the only way to choose the right altitude.
What is the context for your atomics proposal?
None. It's just my library. Similar to TBB.
I dislike the C++0x proposal because it looks too much like plain C, and I remember how an active desire (driven by IBM, I think) to be compatible with plain C ruined another standard, the C++ CORBA mapping (which otherwise might have had things like a sequence template class, and not have suffered debilitating leaks by using something like auto_ptr sources and sinks instead of plain pointers). Here it's not as problematic, but it just looks ugly, and elegance is a real goal, and I would hate it if the current C++0x atomics proposal goes to press like this (it's my civic duty to oppose it).
Maybe you don't need to use the memory-order argument in a switch or anything, but it will still be part of the call (right?), whereas a template argument is completely invisible. Is there any reason not to use a templated function like TBB does, where template specialisation could do the same as having different overloads? There's a whole lot of things that can be done with template metaprogramming... I was thinking of having architecture-specific traits for the operations, and using those to select appropriate "packaging" implementations for the different memory semantics.
First of all, I not a C fan too. I like templates, template metaprogramming etc.
The main reason to make memory model as parameter - compliance with C++0x. It's The Principle Of Least Surprise. It doesn't metter whether it's better or worse, it's what user expects and knows.
At the present I don't see anything considerable what templates can give here. Handling of "inheritance" between fence types will be more complicated with templates.
Please elaborate about "things that can be done with template metaprogramming" wrt atomics/fence types. Can you provide some examples?
Dmitriy V'jukov
