Fact for September 16, 2008 – computational accuracy

Due to the vagaries of rounding, to a calculator or computer that is calculating to, say, eight decimal places,

((\frac{2}{3})+(\frac{2}{3}))-(\frac{1}{3}) \ne (\frac{2}{3})+((\frac{2}{3})-(\frac{1}{3})).

Thus computer arithmetic is not necessarily associative. In this case, while the former ought to come out to 1, the latter will actually be 0.99999997.


Comment (obtuse, impolite, or otherwise "troll"-like comments may be deleted)

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Create a free website or blog at WordPress.com.

Up ↑

%d bloggers like this: