rounding inconsistency

dave_frank · ‎10-15-2003

Will someone confirm below is a problem with CVF6.6B ?

program cvf_rounding_inconsistency
implicit none
real,parameter :: b = 0.002
integer :: a, n

do a = 15320499,15320499
n = a * b
write (*,*) n ! = 30640 truncates
end do

a = 15320499
n = a * b
write (*,*) n ! = 30641 rounds
end program

This problem surfaced when testing a CLF poster's
rounding case complaint for his unnamed compiler.

If I compile with /nooptimize the output truncates
BOTH outputs above to 30640 ??

gfthomas8 · ‎10-15-2003

There's no inconsistency in how CVF rounds, irrespective of optimization.

For floats, the default rounding mode is round to nearest. Rounding mode doesn't apply to integers. Use NINT. NINT(a*b) will give 30641 all the time.

Ciao,
Gerry T.

Steven_L_Intel1 · ‎10-15-2003

Actually, "round to even" is used. See The Perils of Real Numbers, Part 3 for more info.

Steve

gfthomas8 · ‎10-15-2003

Yes, I meant to say that my default is to round to nearest so YMMV.

Ciao,
Gerry T.

gfthomas8 · ‎10-16-2003

Actually, 'round to even' is a loose way of saying take the result whose LSB is 'off' in the event of a tie in 'round to nearest'. From the Feb/2003 IEEE 754 standard:

4.1. Round to Nearest
An implementation of this standard shall provide round to nearest as the default rounding mode. In this mode the representable value nearest to the infinitely precise result shall be delivered; if the two nearest representable values are equally near, the one with its least significant bit zero shall be
delivered....

The CVF docs correctly use the term 'round to nearest' as does F2K.

Ciao,
Gerry T.

dave_frank · ‎10-16-2003

n = a * b ! = 36040
n = a * b ! = 36041

obviously in the 2nd evaluation above
CVF didnt follow the same rule it followed in the 1st
evaluation..
Hence my subject line: rounding inconsistency