[DLSLUG-Discuss] Java fluke, double precision pitfall

[Bill McGonigle]

On Dec 29, 2005, at 04:00, Roger wrote:

> 1.2 minus 1.0 = 0.19999999999999996

You're seeing the floating point precision problem.  The IEEE specifies 
a standard for representing arbitrarily large floating point numbers in 
a 32-bit storage space.  Clearly, something has to give.

Here's a decent overview from Sun:

   http://docs.sun.com/source/806-3568/ncg_goldberg.html#680

And a simple explanation from Microsoft:

   http://support.microsoft.com/kb/214118/EN-US/

Try your code using Integer types and you should get 0 == 0, but you 
have a fixed range of values for the variable.

-Bill

-----
Bill McGonigle, Owner           Work: 603.448.4440
BFC Computing, LLC              Home: 603.448.1668
[e-mail addresses are not shown in archives]           Cell: 603.252.2606
http://www.bfccomputing.com/    Page: 603.442.1833
Blog: http://blog.bfccomputing.com/
VCard: http://bfccomputing.com/vcard/bill.vcf