Community
cancel
Showing results for 
Search instead for 
Did you mean: 
Highlighted
Beginner
6 Views

Problem with Floating point precision (Compaq Visual Fortran 6.6C)

I have following code:
Code:
 real*8 r1,r2,r3
 real*4 r4

 r1=-1.20006351619769219d-0001
 r2=+1.20005790949126556d-0001
 r3=r2+r1
 r4=r3

When I look at the Registers, result of addition is
Code:
ST0 = -5.60670642663000000e-0007


but to the variable r3 is stored -5.606706426630748E-007.
I tried to display these values in bitview:
-5.60670642663000000e-0007 = BEA2D01FC82BFD3E
-5.606706426630748e-007 = BEA2D01FC82C0000
when I try to load variable r3 to FPU (r4=r3) there appears in ST0 register correct value-5.60670642663000000e-0007.
This behavior causes different results in debug and release version of my program and completely different results.
Where can be problem?
0 Kudos
1 Reply
Highlighted
New Contributor I
6 Views

real*8 is 64-bit precision. real*4 is 32-bit precision. when you add the two 64-bit numbers, you get better precision (more digits, less zeroes). when you truncate that to 32-bit then you lose some of the precision and some of the extra digits.
0 Kudos