[DLSLUG-Discuss] Java fluke, double precision pitfall

[Python]

Base 10 fractions can not be stored exactly in binary representations.
Here is the Python equivalent:

>>> .2
0.20000000000000001
>>> str(.2)
'0.2'
>>> 1.2 - 1.0
0.19999999999999996
>>> str(1.2 - 1.0)
'0.2'

Note that the string function rounds sensibly.  Most languages will
provide a means to see the value as it would be (naively) expected to
appear.

On Thu, 2005-12-29 at 08:39 -0500, Anne Stanton wrote:
> This is fairly common in many of the core systems. I noticed it during
> year 2000, when programming had to make changes not to our code, but to
> modules supplied by other vendors. It also causes difficulty when
> reconciling to 0 in some of the more advanced accounting software
> systems. 
Dollar values should be stored as decimals (or fixed point integers).
Binary floating point values can not be expected to equal the decimal
equivalents.  In ternary, 0.1 == 1/3, but can not be expressed exactly
in decimal where you get 0.33333333.

> As you develop new programs add to your plate this awareness. 
> 
> Happy New Year everyone!! 
> 
> 
> Anne Stanton, MVP/CRM, MBA/ACC
> President
> The Norwich Group
> www.thenorwichgroup.com
> 802-526-1023
>  
> 
> -----Original Message-----
> From: [e-mail addresses are not shown in archives]
> [mailto:[e-mail addresses are not shown in archives]] On Behalf Of Roger
> Sent: Thursday, December 29, 2005 4:00 AM
> To: [e-mail addresses are not shown in archives]
> Subject: [DLSLUG-Discuss] Java fluke, double precision pitfall
> 
> 
> Hello everyone,
> 
> Yes, I'm still a Java nut, but...
> 
> While I was getting beaten in a TopCoder contest, I discovered this
> fluke.
> 
> I reduced the "fluke" to a few lines of code as possible.
> When I ran the enclosed program, I got the following output.
> 
> 1.2 minus 1.0 = 0.19999999999999996
> 
> I'm able to find ways around this by type casting and using the
> BigDecimal object. I just thought I would share this random tid bit with
> you guys.
> 
> -Roger
> 
> _______________________________________________
> DLSLUG-Discuss mailing list
> [e-mail addresses are not shown in archives]
> http://dlslug.org/mailman/listinfo/dlslug-discuss
-- 
Lloyd Kvam
Venix Corp