[DLSLUG-Discuss] Java fluke, double precision pitfall
e-mail addresses are not shown in archives
Thu Dec 29 22:19:32 EST 2005
That's kinda interesting.
It seems that the JVM was probably following the IEEE rules; however, when I compile small
equivalent C++ programs on a PowerPC and a Pentium, I get the exact expected base-10 output. I
have a feeling that g++ and the accompanying libraries are going some weird rounding on my behalf
that the JVM's don't do. hmmm....
oh well... I'll be on the lookout for that again whenever I play fast and loose with double
precision math. lol...
I promise to read those links more closely.
--- Bill McGonigle <[e-mail addresses are not shown in archives]> wrote:
> On Dec 29, 2005, at 04:00, Roger wrote:
> > 1.2 minus 1.0 = 0.19999999999999996
> You're seeing the floating point precision problem. The IEEE specifies
> a standard for representing arbitrarily large floating point numbers in
> a 32-bit storage space. Clearly, something has to give.
> Here's a decent overview from Sun:
> And a simple explanation from Microsoft:
> Try your code using Integer types and you should get 0 == 0, but you
> have a fixed range of values for the variable.
> Bill McGonigle, Owner Work: 603.448.4440
> BFC Computing, LLC Home: 603.448.1668
> [e-mail addresses are not shown in archives] Cell: 603.252.2606
> http://www.bfccomputing.com/ Page: 603.442.1833
> Blog: http://blog.bfccomputing.com/
> VCard: http://bfccomputing.com/vcard/bill.vcf
Yahoo! for Good - Make a difference this year.
More information about the DLSLUG-Discuss