[DLSLUG-Discuss] Java fluke, double precision pitfall

[Roger]

Thanks guys,

That's kinda interesting.
It seems that the JVM was probably following the IEEE rules; however, when I compile small
equivalent C++ programs on a PowerPC and a Pentium, I get the exact expected base-10 output. I
have a feeling that g++ and the accompanying libraries are going some weird rounding on my behalf
that the JVM's don't do. hmmm....

oh well... I'll be on the lookout for that again whenever I play fast and loose with double
precision math. lol...
I promise to read those links more closely.
Thanks again.

-Roger


--- Bill McGonigle <[e-mail addresses are not shown in archives]> wrote:

> On Dec 29, 2005, at 04:00, Roger wrote:
> 
> > 1.2 minus 1.0 = 0.19999999999999996
> 
> You're seeing the floating point precision problem.  The IEEE specifies 
> a standard for representing arbitrarily large floating point numbers in 
> a 32-bit storage space.  Clearly, something has to give.
> 
> Here's a decent overview from Sun:
> 
>    http://docs.sun.com/source/806-3568/ncg_goldberg.html#680
> 
> And a simple explanation from Microsoft:
> 
>    http://support.microsoft.com/kb/214118/EN-US/
> 
> Try your code using Integer types and you should get 0 == 0, but you 
> have a fixed range of values for the variable.
> 
> -Bill
> 
> -----
> Bill McGonigle, Owner           Work: 603.448.4440
> BFC Computing, LLC              Home: 603.448.1668
> [e-mail addresses are not shown in archives]           Cell: 603.252.2606
> http://www.bfccomputing.com/    Page: 603.442.1833
> Blog: http://blog.bfccomputing.com/
> VCard: http://bfccomputing.com/vcard/bill.vcf
> 
> 



	
		
__________________________________ 
Yahoo! for Good - Make a difference this year. 
http://brand.yahoo.com/cybergivingweek2005/