0

I understand the IEEE rounding in R for values that are to one decimal place (ie 0.5 becomes 0, 1.5 becomes 2) but what about numbers that are to more than one decimal place. The problem I'm having specifically is that 0.555 is rounded to 0.56 but 579.555 is rounded to 579.55. In the help for round it says that it depends on how the OS represents smaller numbers. I'm using Windows 7, would anybody be able to explain how the representation is not exact for 3 decimal places (I could understand 10 or more but 3 seems like a pretty easy number to represent exactly) and why 0.555 and 579.555 are represented differently.

stat_student
  • 787
  • 10
  • 17
  • 1
    one hint/way forward: compare `print(0.555,digits=20)` to `print(579.555,digits=20)` ... – Ben Bolker May 26 '16 at 17:10
  • 1
    Dupe of [Why are these numbers not equal?](http://stackoverflow.com/q/9508518/903061). If you want exact representations at 3 decimal places, multiply by `10e3` and use integers. – Gregor Thomas May 26 '16 at 17:18
  • That is a nice hint. Is this representation due to R or Windows? – stat_student May 26 '16 at 17:18
  • 1
    Representation is due to double-precision floating point -- generic to almost every computational platform (with weird exceptions like binary-coded decimal). So R more than Windows, but really much more general than R. See links/discussion in the answers to the question that @Gregor linked ... – Ben Bolker May 26 '16 at 17:21

0 Answers0