0

We're used to floats being inaccurate for obvious reasons. I thought Decimals were supposed to be accurate though because they represent all the base 10 digits

This code gives us the right answer of 9

print(27*3/9)

So I thought, oh that's integer multiplication followed by division, that's why it's accurate

But nope, this gives us the correct 9.0:

print(27*(3/9))

So why does

print(Decimal(27)*(Decimal(3)/Decimal(9)))

give the incorrect 8.999999999999999999999999999

I understand that 3/9 is 0.333... which can't be represented as a terminating decimal. But why then is base 2 float accurate?

Alec
  • 8,529
  • 8
  • 37
  • 63
  • 4
    Base 10 isn't magic. Working in base 10 won't save you from rounding errors. It lines up better with human intuition, since humans are used to that base, and it's more suitable for environments with rounding requirements defined in base 10, but that's all. – user2357112 Apr 12 '23 at 21:35
  • 2
    You'll see different rounding errors in different bases, or with different precision levels. Sometimes rounding errors happen to cancel. – user2357112 Apr 12 '23 at 21:39
  • 4
    float isn't accurate you just happen to be lucky with the rounding. try printing `Decimal(3/9)` to see how 1/3 is represented by a float. when you multiply by 27 and the result is rounded to a float you just get `9.` back – Sam Mason Apr 12 '23 at 21:39
  • 4
    `Decimal` isn't an arbitrary precision format, any more than `float` is. It just uses base 10 instead of base 2 to store approximations of arbitrary real numbers. – chepner Apr 12 '23 at 21:39
  • 1
    Does this answer your question? [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – Random Davis Apr 12 '23 at 21:40
  • Python provides the `fractions` module if you want to do exact arithmetic with rational numbers: https://docs.python.org/3/library/fractions.html#module-fractions – slothrop Apr 13 '23 at 09:50

2 Answers2

2

As stated by the comments responding to your question you were just lucky in finding an example that happened to produce an exact answer using floats while the Decimal result was slightly out.

As you pointed out 3/9 is 0.33 recurring so will be approximated by both schemes. In general this will be true as the majority of numbers can't be represented by either system, but for numbers we humans care about Decimal can often be easier to reason about.

A couple of useful tools to help understand what's going on here are that Decimal(float(value)) will give you the full decimal expansion of value and math.nextafter(value, math.inf) will give you next representable float after value.

To see what's going inside the calculation it helps to see the intermediate values as they are calculated. I'd proceed as follows:

from decimal import Decimal as D

print(D(1) / 3, D(1 / 3), sep='\n')

This should print out:

0.3333333333333333333333333333
0.333333333333333314829616256247390992939472198486328125

The decimal output is indeed closer to the correct answer as it is using more state to represent the value.

We can then multiply by 27:

print(D(1) / 3 * 27, D(1 / 3) * 27, sep='\n')

Printing out:

8.999999999999999999999999999
8.999999999999999500399638919

They're now both truncated to the context's precision, but it's possible to "see inside" the intermediate "float" value before it's rounded to the nearest representable values.

Using math.nextafter we can check what nearby values are to understand that the rounding operation is being performed correctly:

from math import nextafter, inf

print(D(nextafter(9, -inf)), D(1 / 3) * 27, D(nextafter(9, inf)), sep='\n')

As you can see the intermediate value in the middle is closer to 9 than either of the nearest representable values. The floating point unit will therefore pick 9 as the result, and is why it appears to give a correct result from this calculation, even though the intermediate value of 1/3 wasn't the closest.

In general the rounding to nearest representable value performed at the end of each floating point operation can be expected to introduce some error but hopefully these tools will help you understand what's going on inside.

Another useful tool provided by modern languages is the hex representation of floats. Python exposes this as a hex() method, but it doesn't help with this question much. I guess you can use it to see that nextafter just changes the LSB of the fractional part.

Sam Mason
  • 15,216
  • 1
  • 41
  • 60
2

We're used to floats being inaccurate for obvious reasons.

The thing is that for the most part, those "obvious reasons" apply to decimal fractions, too.

You can't represent the fraction 1/3 as a decimal fraction. You can approximate it as 0.333, or 0.333333333, but no matter how many 3's you add at the end, it's never going to be exact. And if you multiply it by 3 again, you're liable to get 0.999999999, not 1.0.

You can't represent π = 3.141592654… exactly in either decimal or binary.
You can't represent √2 = 1.41421356… exactly in either decimal or binary.
You can't represent e = 2.718281828… exactly in either decimal or binary.

My point is that neither decimal nor binary has a monopoly on accuracy (or inaccuracy). It only seems like decimal is always right, and binary is often wrong, and the reason for that is just that we're so used to seeing decimal fractions, and we overlook their inaccuracies, but the inaccuracies that arise when we convert to/from binary always startle us.

Now, one way that decimal is "better" than binary is that, mathematically, there are no binary fractions that can't be converted exactly to decimal, while there are plenty of decimal fractions (most of them, actually) that can't be converted exactly to binary. That is, if you've got a binary fraction like 0b1.010101, you can always convert it to an exact decimal fraction 1.328125, but if you've got even the simplest decimal fraction 0.1, when you try to convert it to binary you get an infinitely-repeating pattern 0b0.0001100110011….

But this is all sort of by way of background, and doesn't answer your other question. Why does 27*(3/9) happen to give you an exact answer in binary, but not in decimal, even though 3/9 isn't representable exactly in either decimal or binary? And the answer is just that roundoff error is kind of random, and sometimes, two roundoff errors cancel each other out. In IEEE-754 floating point, which is what Python is probably using, the closest double-precision value to 3/9 is a 53-bit binary fraction which works out to exactly 0.333333333333333314829616256247390992939472198486328125 . When you multiply that number by 27, the exact answer would be 8.999999999999999500399638918679556809365749359130859375. IEEE-754 says that when you multiply, the result you get (if inexact) must be a correctly-rounded version of the exact result, and that number is close enough to 9.0 that it does indeed get rounded up.

I'm not sure how Python's Decimal type is implemented. Either it doesn't have the same rounded-actual-result guarantee, or it ends up happening that the exact result (in Decimal) is closer to 8.999999999999999999999999999 than it is to 9.0.

Footnote: I said that "roundoff error is kind of random", but that's not really true. A number theorist could tell us exactly which results are going to be exact and which are approximate, could tell us exactly when two roundoff errors would cancel each out and when they would persist. But I don't know enough about number theory to even try to make that argument.

Steve Summit
  • 45,437
  • 7
  • 70
  • 103
  • AFAIK, Python's decimal module is an implementation of IEEE754 decimal arithmetic and does implement rounding correctly. 3 * 27 is 81, so there will be a trailing 1 which was rounded down – Sam Mason Apr 14 '23 at 12:44