[DLSLUG-Discuss] Java fluke, double precision pitfall
e-mail addresses are not shown in archives
Thu Dec 29 10:29:04 EST 2005
Base 10 fractions can not be stored exactly in binary representations.
Here is the Python equivalent:
>>> 1.2 - 1.0
>>> str(1.2 - 1.0)
Note that the string function rounds sensibly. Most languages will
provide a means to see the value as it would be (naively) expected to
On Thu, 2005-12-29 at 08:39 -0500, Anne Stanton wrote:
> This is fairly common in many of the core systems. I noticed it during
> year 2000, when programming had to make changes not to our code, but to
> modules supplied by other vendors. It also causes difficulty when
> reconciling to 0 in some of the more advanced accounting software
Dollar values should be stored as decimals (or fixed point integers).
Binary floating point values can not be expected to equal the decimal
equivalents. In ternary, 0.1 == 1/3, but can not be expressed exactly
in decimal where you get 0.33333333.
> As you develop new programs add to your plate this awareness.
> Happy New Year everyone!!
> Anne Stanton, MVP/CRM, MBA/ACC
> The Norwich Group
> -----Original Message-----
> From: [e-mail addresses are not shown in archives]
> [mailto:[e-mail addresses are not shown in archives]] On Behalf Of Roger
> Sent: Thursday, December 29, 2005 4:00 AM
> To: [e-mail addresses are not shown in archives]
> Subject: [DLSLUG-Discuss] Java fluke, double precision pitfall
> Hello everyone,
> Yes, I'm still a Java nut, but...
> While I was getting beaten in a TopCoder contest, I discovered this
> I reduced the "fluke" to a few lines of code as possible.
> When I ran the enclosed program, I got the following output.
> 1.2 minus 1.0 = 0.19999999999999996
> I'm able to find ways around this by type casting and using the
> BigDecimal object. I just thought I would share this random tid bit with
> you guys.
> DLSLUG-Discuss mailing list
> [e-mail addresses are not shown in archives]
More information about the DLSLUG-Discuss