11

Possible Duplicate:
Haskell ranges and floats

Why does the following output occur in haskel:

[0.1,0.3..1]
[0.1,0.3,0.5,0.7,0.8999999999999999,1.0999999999999999]
  1. What is the math behind the 1.0999999999999999 (I am on a 64bit linux machine if its useful)?
  2. Why doesnt it stop at 0.8999999999999999 when obviously 1.0999999999999999 is out of range?
Community
  • 1
  • 1
footy
  • 5,803
  • 13
  • 48
  • 96
  • 2
    As for 1), what you're seeing is just the inaccurate representation of the decimal number 1.1 as a floating point number – Niklas B. Nov 02 '12 at 21:59
  • @NiklasB. Still it doesnt explain why the inaccuracy in the first place if 0.3-0.1 is a clear 0.2 which is a rational number easily written in a floating point number – footy Nov 02 '12 at 22:00
  • Where I come from, 0.3-0.1 is a clear 0.2, and 0.9 + 0.2 == 1.1. Keep in mind that you are *not* using rationals here, you are using floating point, which cannot represent arbitrary rationals accurately. You are mistaken in believing that 0.1 can be represented accuratly in the binary system. – Niklas B. Nov 02 '12 at 22:01
  • @NiklasB. it was a typo... however, floating point 0.3 - 0.1 is not a problem to compute accurately right? – footy Nov 02 '12 at 22:03
  • no - even .1 cannot be represented exactly with a floating point number – ErikR Nov 02 '12 at 22:05
  • @user5402 0.1 can be represented accurately. – footy Nov 02 '12 at 22:22
  • 2
    @footy This is not a Haskell problem, it is a computer language problem in general. See http://floating-point-gui.de/basic/ – Mark Thomas Nov 02 '12 at 22:27
  • @footy No, you cannot represent `0.1` exactly in binary, e.g. [see here](http://en.wikipedia.org/wiki/Binary_numeral_system#Fractions_in_binary). – Chris Taylor Nov 02 '12 at 22:27
  • 1
    If `f1` is the closest floating point number to `m1/n` and `f2` is the closest floating point number to `m2/n`, it does NOT follow that `f1+f2` is the closest floating point number to `(m1+m2)/n`. And no, no IEEE floating point numbers are exactly equal to 0.1 or 0.2. – aschepler Nov 02 '12 at 22:27
  • @MarkThomas The relevancy to Haskell is why the return value includes `1.09999...` at all, when it looks like the user is requesting numbers only up to `1.0`. – Chris Taylor Nov 02 '12 at 22:28
  • @ChrisTaylor http://en.wikipedia.org/wiki/IEEE_floating_point shows that 0.1 can be easily represented right? What am I missing? – footy Nov 02 '12 at 22:30
  • @footy It can't show that, since it simply isn't true. Can you link to the precise bit of the IEEE standard (or the Wikipedia article about the standard) that you believe shows that `0.1` has an exact decimal representation? – Chris Taylor Nov 02 '12 at 22:34
  • 1
    @ChrisTaylor .. now I understand! Thanks. For 0.1 its a recursive representation of .00011 in binary. Hence its an irrational number in binary! – footy Nov 02 '12 at 22:46
  • 5
    @footy No, all periodic fractions are rational in all bases. It's a _nonterminating_ binary fraction, though. – Daniel Fischer Nov 02 '12 at 22:56
  • @ChrisTaylor: IEEE 754 defines both binary and decimal floating point. It is incorrect to assert that floating point cannot represent .1 exactly. If you are explaining to somebody that the particular binary floating point implementation they are using cannot represent .1 accurately, you need to explain that it is binary. – Eric Postpischil Nov 03 '12 at 12:26

2 Answers2

19

Why the overshoot?

[0.1,0.3..1] is short for enumFromThenTo 0.1 0.3 1.0

The Haskell report says

For Float and Double, the semantics of the enumFrom family is given by the rules for Int above, except that the list terminates when the elements become greater than e3 + i∕2 for positive increment i, or when they become less than e3 + i∕2 for negative i.

Here e3 = 1.0 and your increment i = 0.2, so e3 + i∕2 = 1.1. It's only supposed to stop when it goes bigger than that.

You asked it to stop at 1, but it can only stop at 0.9 or 1.1. There's a rounding error (floating types are inherently inaccurate) and 1.1 has ended up as 1.09999999999, so since this is not greater than 1.0 + i/2, it's allowed.

In fact, even if it were equal to 1.0+i/2 it's allowed, as you can check using the exact [0.1,0.3..1]::[Rational] (after importing Data.Ratio).

You can avoid the problem by calculating the upper limit you're aiming for, 0.9, and specifying that: [0.1,0.3..0.9]. You won't suffer from rounding error unless your increment is small and your numbers are large, i.e. you're working beyond the accuracy of Double for large numbers.

Why the inaccuracy?

1.09 recurring is mathematically indistinguishable from 1.1, but here we have a finite number of 9s, and that's strictly less than 1.1.

Floating point numbers are stored as if they're in scientific notation, eg 4.563347x10^-7, but in binary, so like 01.1001110101x2^01101110.

This means your number can only be stored completely accurately as a Float if you can express it by summing powers of two, just as you can only write a number in decimal if you can express is by summing powers of 10.

0.2 in your example is 0.001100110011 in binary, with the 0011 repeating forever and 1.1 is 1.0001100110011 again with 0011 repeating forever.

Since only a finite part of those will be stored, when converted back to decimal to show you, they'll be a little out. Often the difference is so small it gets rounded away again, but sometimes you can see it, as here.

This inherent inaccuracy is why enumFromThenTo lets you go above the top number - it's stopping you from having too few because of rounding errors.

David G
  • 94,763
  • 41
  • 167
  • 253
AndrewC
  • 32,300
  • 7
  • 79
  • 115
  • The Haskell standard is broken. It used to be correct, but it was changed. – augustss Nov 03 '12 at 01:54
  • @augustss: Can you elaborate? – Niklas B. Nov 03 '12 at 01:54
  • The difference between the real floating point value and the desired mathematical entity doesn't get rounded away; it's just often not displayed on rendering as decimals. – Donal Fellows Nov 03 '12 at 09:58
  • @DonalFellows That's exactly what I meant by "converted back to decimal to show you...so small it gets rounded away again" meaning that there's a second rounding error on conversion to decimal _for the purposes of show_. I didn't mean to mislead anyone in to thinking the underlying floating point got edited in any way - this is Haskell after all and we don't do destructive update when applying the show function! – AndrewC Nov 03 '12 at 11:09
  • Well, we all know that Float and Double are best handled with care (and not to assume too much about their instances). On the other hand I'm a bit miffed with the behavior of Rational which is after all exact and for which there is no good reason to overshoot in their Enum instance. – Jedai Nov 05 '12 at 12:00
  • I agree. There's no reason to overshoot. Let me hit the character limit to say that even 1.09999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999999 is stored exactly in Rational – AndrewC Nov 05 '12 at 13:06
  • 1
    This is years late, but just in case some time traveler visits here in the future: the "Why the inaccuracy?" section is not correct. There really is an inaccuracy here: the number showed to you, and the closest `Double` (or `Float`) to `1.1`, are not the same. You can verify this by entering `0.1 + 0.2 + 0.2 + 0.2 + 0.2 + 0.2 == 1.1` (the left hand side is the calculation that produces `1.0999999999999999`) in ghci, which will reply `False`. It is not *just* an infelicity in conversion to decimal. More details [here](https://stackoverflow.com/q/588004/791604). – Daniel Wagner Dec 18 '20 at 06:38
9

The simple answer

To understand this behaviour, you need to know that the expression [a,b..c] will be desugared into enumFromThenTo a b c where enumFromThenTo is a method of the Enum class.

The Haskell standard says that

For Float and Double, the semantics of the enumFrom family is given by the rules for Int above, except that the list terminates when the elements become greater than e3 + i∕2 for positive increment i, or when they become less than e3 + i∕2 for negative i.

Standards are standards, after all. But that's not very satisfactory.

Going deeper

The Double instance of Enum is defined in the module GHC.Float, so let's look there. We find:

instance Enum Double where
  enumFromThenTo = numericFromThenTo

That's not incredibly helpful, but a quick Google search reveals that numericFromThenTo is defined in GHC.Real, so let's go there:

numericEnumFromThenTo e1 e2 e3 = takeWhile pred (numericEnumFromThen e1 e2)
                                where
                                 mid = (e2 - e1) / 2
                                 pred | e2 >= e1  = (<= e3 + mid)
                                      | otherwise = (>= e3 + mid)

That's a bit better. If we assume a sensible definition of numericEnumFromThen, then calling

numericEnumFromThenTo 0.1 0.3 1.0

will result in

takeWhile pred [0.1, 0.3, 0.5, 0.7, 0.9, 1.1, 1.3 ...]

Since e2 > e1, the definition of pred is

pred = (<= e3 + mid)
  where
    mid = (e2 - e1) / 2

Therefore we will take elements (call them x) from this list as long as they satisfy x <= e3 + mid. Let's ask GHCi what that value is:

>> let (e1, e2, e3) = (0.1, 0.3, 1.0)
>> let mid = (e2 - e1) / 2
>> e3 + mid
1.1

That's why you see 1.09999... in the list of results.

The reason that you see 1.0999... instead of 1.1 is because 1.1 is not representable exactly in binary.

The reasoning

Why would the standard prescribe such bizarre behaviour? Well, consider what might happen if you only took numbers that satisfied (<= e3). Because of floating point error or nonrepresentability, e3 may never appear in the list of generated numbers at all, which could mean that innocuous expressions like

[0.0,0.02 .. 0.1]

would result in

[0.0, 0.02, 0.04, 0.06, 0.08]

which seems a little bizarre. Because of the correction in numericFromThenTo, we make sure that we get the expected result for this (presumably more common) use case.

Chris Taylor
  • 46,912
  • 15
  • 110
  • 154
  • 1
    just a sidenote: `[0.0,0.02..0.1]` ==> `[0.0,2.0e-2,4.0e-2,6.0e-2,7.999999999999999e-2,9.999999999999998e-2]` ; `[1.1,1.3..2.1]` ==> `[1.1,1.3,1.5,1.7,1.9,2.0999999999999996]` ; `[11.1,11.3..12.1]` ==> `[11.1,11.3,11.500000000000002,11.700000000000003,11.900000000000004,12.100000000000005]`. – Will Ness Nov 03 '12 at 08:10