Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.
2464 Discussions

Atomic floats, is this implementation safe?

robert_jay_gould
Beginner
1,712 Views

I made an atomic float, and it's probably not blazingly fast (that's ok), but its faster than wrapping a lock around a float, and it works, but I'm not sure if this is because of my good luck, or if this is actually thread safe. I think it is... but you never know, so I came to ask the experts :)

struct AtomicFloat: public tbb::atomic

{

float compare_and_swap(float value, float compare)

{

size_t value_ = tbb::atomic::compare_and_swap((size_t&)value,(size_t&)compare);

return reinterpret_cast(value_);

}

float operator=(float value)

{

size_t value_ = (*this).tbb::atomic::operator =((size_t&)value);

return reinterpret_cast(value_);

}

operator float()

{

return reinterpret_cast(*this);

}

float operator+=(float value)

{

volatile float old_value_, new_value_;

do

{

old_value_ = reinterpret_cast(*this);

new_value_ = old_value_ + value;

} while(compare_and_swap(new_value_,old_value_) != old_value_);

return (new_value_);

}

};

Also as a caveat I'm placing a static assert for size_of(float) == size_of(size_t).

Thanks!

0 Kudos
53 Replies
RafSchietekat
Valued Contributor III
170 Views

"Anyway, I'm not set on getting somewhere with this, just considering the possibilities for the moment." But let's narrow it down to my favourite (and you might have guessed because I named it last)...

If an atomic floating-point type does not get to have compare_and_swap (which should probably be deprecated everywhere anyway), there is no urgent need to let minus zero pass for plus zero or vice versa (by repeated attempts with the respective value representations). It is not quite the same as spurious failure, because there you have an assumed progress guarantee ("if at first you don't succeed, try, try again"), but would a programmer really be tempted to use the same comparand every time with a floating-point type? If an atomic is used for a spin lock, it is an obvious use case to repeatedly try to substitute 1 for 0, so it helps that there is a one-to-one mapping between values (all equal to themselves and different from any other) and value representations for integral types (assuming that the compiler has done its job of normalising "true" for "bool"). But with a floating-point type, the most prominent use case is to read a value, perform a computation on it, and write back the result, and try again with the new snapshot instead if the substitution failed; it does not seem so bad to have to do this for the occasional failure based on plus/minus zero, and, because there would be no need for comparisons, no complexities would be introduced related to plus/minus zero and NaN.

Did I overlook anything?

0 Kudos
jimdempseyatthecove
Honored Contributor III
170 Views

>>But with a floating-point type, the most prominent use case is to read a value, perform a computation on it, and write back the result, and try again with the new snapshot instead if the substitution failed; it does not seem so bad to have to do this for the occasional failure based on plus/minus zero, and, because there would be no need for comparisons, no complexities would be introduced related to plus/minus zero and NaN.

Did I overlook anything?<<

If you fetch and savethe old value of the floating point variable using float context, the compiler may use floating point type instructions to copy the data into the "old_value" (especially since it sees you performing the reduction (+) one line later). This is bad news if you use that value for the compare and exchange .AND. if the old_value was tidied up in the process of copied. In this case you will never pass your compare_and_exchange. Pseudo code of the correct way

float* pCell = get_address_of_index(index); // vector cannot move
do {
float old_value;
(DWORD)old_value = *((DWORD*)pCell); // using 32-bit registers
float new_value = old_value + bump_value; // using floating point registers
} while(!DCAS(pCell, new_value, old_value)); // using 32-bit registers

Notes, check your flavor of DCAS for order and kind of variables
The cell pointer must remain valid for the duration of theatomic reduction.
Caution relating to vector >
Some forms of containers might not provide persistance of vector
cells accross vector expansion due to other thread potentially expanding
vector during insertion. To protect against this youmay need additional
coding (locks or other defensive coding).
Ithink that tbb containers do not move a cellonce the cell
isused whereasother containers make no such guarantee.

Jim Dempsey



0 Kudos
jimdempseyatthecove
Honored Contributor III
170 Views

Additionally you my need to declare the old value with volatile (depending on optimization of your compiler).

Or if using Intel C++ consider using __ld4_acq(void* src);

Jim Dempsey

0 Kudos
robert_jay_gould
Beginner
170 Views

Additionally you my need to declare the old value with volatile (depending on optimization of your compiler).

Or if using Intel C++ consider using __ld4_acq(void* src);

Jim Dempsey

Yes its a good idea to add the volatile to "play it safe".

As for vector > In my case I have assurance that vectors aren't modified while work takes place (pipeline-like stages work-tidy up-work)

0 Kudos
RafSchietekat
Valued Contributor III
170 Views

"tidied up in the process of copied" So the user will probably have to be instructed to always include at least two successive compare_and_store() instructions in each iteration if a back-off is included in the loop. Would compare_and_store()'s update by reference also be compromised if the compiler can look inside the implementation? Still, that would merely be annoying (the implementation can be hidden in an object file instead of inlined). BTW, in the case of an IEEE 754 implementation there would be no "tidy up" of numbers (each has a unique value representation, except for the two related to plus/minus zero), so something like "disturb" seems more appropriate.

"Pseudo code of the correct way" That's what atomic would get rid of, or else what's the point...

0 Kudos
jimdempseyatthecove
Honored Contributor III
170 Views

In addition to -0, unnormalized numbers may get tidied up too.

An atomic by itself, or as a cell in a standard array, or member variable in structure are all fine as you can safely produce the & (address of) the atomic (for use in the compare_and_swap). If you abstract the further by placingthe atomicinto a container (vector >) and if the vector implementation does not assure that once a cell is used it won't move, or if the container does not have a "do not move this cell while I hold a reference to the cell", then it will not be safe to perform an exchange or store without additional protocol to avoid you trashing un-owned memory. In the case of the atomic reduction operator the actual reduction (atomic operation) would be best placed inside the container as a operator. The container code can be written to work around issues relating to relocating cells while reference pending for purpose of compare_and_swap, swap, or store.

Jim Dempsey

0 Kudos
RafSchietekat
Valued Contributor III
170 Views

"In addition to -0, unnormalized numbers may get tidied up too." Hath not a denormalised number a specific value? If you "tidy it up", does it not change? (Sorry, couldn't resist.)

0 Kudos
RafSchietekat
Valued Contributor III
170 Views

Would the following set of features be acceptable with float and double: {compare_and_,fetch_and_,}store(), operator=, load(), operator value_type(), operator{+=,-=,*=,/=}?

compare_and_swap() needs to be omitted because it would be fraught with problems, as already discussed.

Integer types have their use even for operations like compare_and_or(), but with floating points it seems better to wipe the slate clean than to add, for consistency, numerous operations referring to multiplication and division.

Still, it does not seem fully satisfactory: maybe operators ++ and -- should still be available. But then why not have ++ for bool? And *= and /= for integers? And what about >>= and <<=? But adding is always nicer than taking away, so...

0 Kudos
jimdempseyatthecove
Honored Contributor III
170 Views
Quoting - Raf Schietekat

Would the following set of features be acceptable with float and double: {compare_and_,fetch_and_,}store(), operator=, load(), operator value_type(), operator{+=,-=,*=,/=}?

compare_and_swap() needs to be omitted because it would be fraught with problems, as already discussed.

Integer types have their use even for operations like compare_and_or(), but with floating points it seems better to wipe the slate clean than to add, for consistency, numerous operations referring to multiplication and division.

Still, it does not seem fully satisfactory: maybe operators ++ and -- should still be available. But then why not have ++ for bool? And *= and /= for integers? And what about >>= and <<=? But adding is always nicer than taking away, so...

There is nothing wrong with a compare and swap on floating point data as long as the comparand is obtained using register moves (as opposed to floating point load and store). Example of atomic add for double

double AtomicAdd(double* pd, double d)
{
union
{
LONGLONG iOld;
double dOld;
};
union
{
LONGLONG iNew;
double dNew;
};
while(true)
{
iOld = *(__int64*)pd; // current old value
dNew = dOld + d;
if(InterlockedCompareExchange64(
(volatile LONGLONG*)pd, // loc
iNew, // xchg
iOld) // cmp
== iOld)
return dNew;
}
}

Jim Dempsey

0 Kudos
jimdempseyatthecove
Honored Contributor III
170 Views
Quoting - Raf Schietekat

Would the following set of features be acceptable with float and double: {compare_and_,fetch_and_,}store(), operator=, load(), operator value_type(), operator{+=,-=,*=,/=}?

compare_and_swap() needs to be omitted because it would be fraught with problems, as already discussed.

Integer types have their use even for operations like compare_and_or(), but with floating points it seems better to wipe the slate clean than to add, for consistency, numerous operations referring to multiplication and division.

Still, it does not seem fully satisfactory: maybe operators ++ and -- should still be available. But then why not have ++ for bool? And *= and /= for integers? And what about >>= and <<=? But adding is always nicer than taking away, so...

for float and double operators +=, -=, ++, --, *=, /= are understandable

But you may use for &=, |=, ^= operatorsthe binary operators on the binary values of the floating point number. Examples of use are flipping the sign bit, forcing the sign bit to 1/0, truncating some number of least significant bits, saving some number of the least significant bits, rounding to a specific least significant bit, etc...

For <<= and >>= this would be subjective. It could be a power of two shifting, a power of 10 shifting, power and rooting.

Jim Dempsey

0 Kudos
RafSchietekat
Valued Contributor III
170 Views

"There is nothing wrong with a compare and swap on floating point data as long as the comparand is obtained using register moves (as opposed to floating point load and store)." The idea is to hide these details.

It does not seem appropriate to provide operators that are not available on the underlying fundamental types, but whoever wishes to do something like that (and takes the responsibility for dealing with the binary representation of the floating-point types) will have compare_and_store() as a building block.

0 Kudos
RafSchietekat
Valued Contributor III
170 Views
0 Kudos
Reply