Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Mute
- Printer Friendly Page

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

10-30-2013
11:14 PM

210 Views

precision issues

Hello Sir,

I have a precision issue with the below code. If I do the calculations for the same input in my calculator I get -13421772.8

Whereas with compiler I get -13421773.0, and this is a considerable difference for us.

The variable used for the above observation is ‘tmp’.

Please help us in resolving this.

Thanks in-advance.

void convert(__m128 &vrz /*inout*/, int art)

{

unsigned int _rounding_mode;

if(1)

{

_rounding_mode = _MM_GET_ROUNDING_MODE();

_MM_SET_ROUNDING_MODE(_MM_ROUND_TOWARD_ZERO);

}

__m128 tmp, scale_vr;

const float scale = (float)((unsigned int)1<<(31-(art)));

scale_vr = _mm_set1_ps(scale);

tmp = _mm_mul_ps(vrz, scale_vr);

vrz = _mm_insert_ps(vrz, _mm_castsi128_ps(_mm_cvtps_epi32(tmp)) , ((1)<<6) | ((1)<<4));

if(1)

{

_MM_SET_ROUNDING_MODE(_rounding_mode);

}

}

void main()

{

float a =( float) -0.8;

m128 vrz;

vrz = _mm_set1_ps(a);

Convert(vrz,7)

}

Thanks,

Eswar Reddy K

1 Solution

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
05:58 AM

210 Views

Link Copied

15 Replies

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-01-2013
02:25 PM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-07-2013
02:01 AM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-07-2013
08:30 PM

210 Views

sorry display proble... below are my compiler options:

WarningLevel: Level3

Optimization: Disabled

UseProcessorExtensions:AVX2

BasicRuntimeChecks : Default

AdditionalOptions : /fp:precise

FlushDenormalResultsToZero : false

FloatingPointModel: Precise

FloatingPointExpressionEvaluation: Default

jimdempseyatthecove

Black Belt

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-11-2013
05:35 AM

210 Views

Eswar,

At issue here may be:

float a = (float)-0.8;

Where a does not use the same rounding mode (round down). As a quick test, compile as Debug build. After setting a=, open a Memory window and examine "&a". View as unsigned 1-byte integer. You should see "205 204 76 191". Subtract 1 from the 205 to undo the round up. Had this been zero, then 0-1 produces 255 with borrow propigating to next byte (i.e. subtract 1 from next byte). There will be some cases where the exponent will need to be adjusted, but this is not necessary for this experiment.

Once the value of a has been adjusted, continue and check the result.

For a formal fix, you will have to be careful as to how you preset your parameters that contain fractional values that cannot be precisely represented in binary. 0.1 is one such fraction as is 0.8.

Jim Dempsey

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-12-2013
06:02 PM

210 Views

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-12-2013
06:12 PM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-12-2013
06:18 PM

210 Views

Thanks Sergey & Jim !

I have obsrved same behaviour irrespective of the configuration.

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-12-2013
10:10 PM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-12-2013
10:53 PM

210 Views

Sergey Kostrov,

The results looks ok for test cases 5 & 6.

Can please provide compiler options and other options if any for the test cases 5 & 6

Thanks,

Eswar Reddy K

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
05:58 AM

211 Views

Bernard

Black Belt

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
06:03 AM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
07:10 AM

210 Views

Thank you!

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
09:01 PM

210 Views

SergeyKostrov

Valued Contributor II

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
09:21 PM

210 Views

Eswar_Reddy_K_

Beginner

- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report Inappropriate Content

11-13-2013
09:26 PM

210 Views

Thanks for the insight.

Topic Options

- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page

For more complete information about compiler optimizations, see our Optimization Notice.