- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Hi all,

As you might know, AVX does not provide instructions for integer types, which are planned to arrive with AVX2. I have a code written using AVX instructions, which basically use _mm256_*_pd() variants of instructions that operate on double-precision floating-point values (the instructions I use are min, max, shuffle, blend, load, loadu, etc.). However my data is actually integers, which I load by casting integer pointers to double pointers, i.e. __m256d reg = _mm256_loadu_pd((double*)intPtr) etc. Functionality wise the code seems to do what I expect, i.e. sorts the data. However, as I haven't tested with all sorts of different data, I'm concerned whether the output will always be correct. What corner cases should I be concerned with? Would the comparisons will always be correct or will there be some integer values where the AVX floating point comparison would not work?

Thanks for comments and suggestions

Link Copied

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including itself.Thus, comparisons involving integers whose bit pattern matches that of a floating-point NaN would be problematic.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**int-to-double**cast in order to use AVX, however... >>...Would the comparisons will always be correct or will there be some integer values where the AVX floating point comparison >>would not work? I would be very carefull because your processing will be dependent on limitation of IEEE 754 Standard and, as recommended in many-many sources, a comparison with an Epsilon could be added ( expect a performance impact ). If your tests are deterministic ( No Random data ) an accuracy of processings, I mean based in integers and then based on doubles, could be verified as soon as both outputs are saved. There are single- and double-precision binary format viewers on the web and you could look / verify how some integer values will look like after conversion to double type.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**[ Test-case ]**... int iIsNan = 0; double dValue = -1.0; double dValueLn = 0.0L; unsigned __int64 iValue = 0U; printf( "dValue = %f\n", dValue ); printf( "dValueLn = %f\n", dValueLn ); printf( "iValue = %I64d\n", iValue ); dValueLn = CrtLog( dValue ); printf( "dValueLn = %f\n", dValueLn ); iValue = ( __int64 )dValueLn; printf( "iValue = %I64d\n", iValue ); iIsNan = _isnan( dValueLn ); if( iIsNan == 0 ) printf( "dValueLn is Not NaN\n" ); else printf( "dValueLn is NaN\n" ); dValue = ( double )iValue; printf( "dValue = %f\n", dValue ); iValue = 9223372036854775800i64; dValue = 0.0L; printf( "iValue = %I64d\n", iValue ); printf( "dValue = %f\n", dValue ); dValue = ( double )iValue; printf( "dValue = %f\n", dValue ); iIsNan = _isnan( dValue ); if( iIsNan == 0 ) printf( "dValue is Not NaN\n" ); else printf( "dValue is NaN\n" ); ...

**[ Output ]**dValue = -1.000000 dValueLn = 0.000000 iValue = 0 dValueLn = -1.#IND00 iValue = -9223372036854775808 dValueLn is NaN dValue = 9223372036854775800.000000 iValue = 9223372036854775800 dValue = 0.000000 dValue = 9223372036854775800.000000

**dValue is Not NaN**Please let me know if you find any problems with the test-case. Best regards, Sergey { UPDATED }Fixed: printf( "iValue = %f\n", iValue ); to printf( "iValue = %I64d\n", iValue );

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**[ Output ]**Test-Case 1 dValue = -1.000000 dValueLn = 0.000000 iValue = 0 dValueLn = -1.#IND00 iValue = -9223372036854775808 dValueLn is NaN dValue = 9223372036854775800.000000 Verifications for Boundary values ( signed and unsigned ) of 64-bit range: Test-Case 2.1 iValueS = 9223372036854775807 dValue = 0.000000 dValue = 9223372036854775800.000000 dValue is Not NaN Test-Case 2.2 iValueS = -9223372036854775808 dValue = 0.000000 dValue = -9223372036854775800.000000 dValue is Not NaN Test-Case 2.3 iValueU = 9223372036854775807 dValue = 0.000000 dValue = 9223372036854775800.000000 dValue is Not NaN Test-Case 2.4 iValueU = 0 dValue = 0.000000 dValue = 0.000000 dValue is Not NaN I'll post source codes of my quick test later after additional verification.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**9223372036854775807**>>... >>dValue =

**9223372036854775800**.000000 >>... Thanks Patrick for the comment!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**some 64-bit integer might have bit pattern of NaN**and might result in an incorrect result... I'll do a couple of tests and I'll be back. Thanks guys for that really nice discussion!

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

I'm very surprized when Intel engineers make some statements without any real verification(s)...Perhaps you missed this part of the original post: However my data is actually integers, which I load by

*casting integer pointers to double pointers*... If one of those "doubles" now points to 64 bits which has the long int value 92211202370041090560 (= 0x7ff8000000000000), it will be intepreted as a (quiet) NaN, and it will compare as "unordered" with any other value.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**704**1090560 and not 922112023

**7004**1090560.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**[ 'double' with NaN value ]**... 00 00 00 00 00 00 f8 ff ...

**[ '__int64' after assignment from 'double' with NaN value ]**... 00 00 00 00 00 00 00 80 ... So, it looks like a developer should watch out for a 0xfff8000000000000 or 18444492273895866368 value. No and let me continue. Next, if a developer converts it back to 'double' it will get 0x43efff0000000000 or 4895411695440101376 and that is done by a C++ compiler (!). It looks like a magic but actually there are No any uncertanties here because only 53 bits (!) will be copied into mantissa and a part of 64-bit integer which is "responsible" for a NaN-code won't be re-created in the 'double'. So, this is

**not**possible to create a NaN value in a double precision variable from a 64-bit integer variable by doing a simple cast, like: ... dValueLn = 0; iNaNIntValue = 18444492273895866368ULL; dValueLn = ( double )iNaNIntValue; ... unless a developer copies these 8 bytes with a 'memcpy' CRT function directly.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

**[ Output ]**... Test-Case 1.3 dValueLn is NaN ... Once again, this is

**not**possible to create a NaN value in a double precision variable from a 64-bit integer variable by doing a simple cast.

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page