5

There seems to be some kind of obscure rounding error when I run the following code:

int roundedTotal = (int)(PriorityJob * 100.0);

Initially PriorityJob = 1.4 and roundedTotal is undefined. Evaluating PriorityJob * 100.0 at that point gives 140. Afterwards roundedTotal = 139.

Apparently, 140.0 is being interpreted as 139.99999. Is this a deficiency in the floating point engine? I have never seen anything like it.

Michael Sandler
  • 1,290
  • 3
  • 17
  • 30
  • 4
    1.4 does not exist as a float. In fact, there's an infinite number of numbers even between 1.0 and 2.0 that cannot be represented as a float, 1.4 is one of them. So it gets stored as an approximation of 1.4 , as 1.3999999999999999 – nos Sep 14 '12 at 12:42
  • @LuchianGrigore It seems like such a simple operation. Why should it fail like that? – Michael Sandler Sep 14 '12 at 12:43
  • 9
    [What every computer scientist should know about floating point arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) – SingerOfTheFall Sep 14 '12 at 12:44
  • 1
    @MichaelSandler: Because a binary floating point format can't represent most decimal fractions exactly, so you have to expect rounding errors. – Mike Seymour Sep 14 '12 at 12:47
  • `roundTotal` is never undefined, since you initialize it! – Kerrek SB Sep 14 '12 at 12:48
  • 4
    @LuchianGrigore: Apparently, he hasn't, so why bother to ask? That's why he's here. Not? – Sebastian Mach Sep 14 '12 at 12:49
  • @nos Of course! Thank you. An elementary oversight on my part. 1.4 is not neatly represented in binary. Care to answer? – Michael Sandler Sep 14 '12 at 12:50
  • 1
    @MichaelSandler it doesn't fail. Casting (especially in old style C-manner) means "I know, it is not the right thing, what I'm doing, but I know all side-effects of this operation and want exactly this result." So why don't you use rounding? e.g.: http://stackoverflow.com/questions/9695329/c-how-to-round-a-double-to-an-int – Valentin H Sep 14 '12 at 13:00
  • 1
    @ValentinHeinitz - the cast is irrelevant here; it just tells the compiler to do what it would do anyway. – Pete Becker Sep 14 '12 at 13:04

1 Answers1

10

Just about every modern computer uses a binary representation for floating-point numbers.

Just as 1/3 = 0.33333333... can't be represented exactly as a decimal fraction, so 1/10 (and hence most non-integer decimal values, including 1.4) can't be represented exactly as a binary fraction. It will instead be represented by the nearest representable value, which may be slightly more or less than the "true" value.

You might want to round to the nearest integer instead: (int)(PriorityJob * 100.0 + 0.5)

Mike Seymour
  • 249,747
  • 28
  • 448
  • 644
  • 2
    And the difference in the displayed values comes up because `std::cout << PriorityJob * 100.0` rounds, while `int roundTotal = PriorityJob * 100.0` drops the fractional part. – Pete Becker Sep 14 '12 at 13:05