0

I want to be able to subtract 1/3 from my variable in a mathematically correct way, meaning: If a 1D is subtracted by 1/3 three times, it should be 0D.

However, I keep ending up with some tiny value (1.1102230246251565E-16)

I've already tried rounding it after each subtraction (Math.round(input*1000000.0)/1000000.0), however I end up with 0.333334 after the second subtraction, and with 1.0E-6 after the third subtraction. Why?

RazorHail
  • 431
  • 1
  • 5
  • 17
  • See also the classic https://stackoverflow.com/q/588004/3001761 – jonrsharpe Nov 23 '17 at 13:44
  • round the result up. Use round function – Benas Nov 23 '17 at 13:45
  • There is no finite representation of 1/3 as double. And double has only a limit number of digits. – Ralf Renz Nov 23 '17 at 13:45
  • how do I round up/down dynamically, based on the x-th decimal-point? – RazorHail Nov 23 '17 at 13:50
  • That is a different question. Ask it as a different Question, and be sure to express yourself more clearly than that. (What do you actually mean by "round up/down dynamically, based on the x-th decimal-point" ... bearing in mind that `float` and `double` are not "decimal" representations. – Stephen C Nov 23 '17 at 13:54
  • You could also have a look at [this other SO question](https://stackoverflow.com/q/5442640/3545273) if you are interested in doing *exact* operations on rational numbers. – Serge Ballesta Nov 23 '17 at 14:11
  • If you want exact arithmetic for arbitrary rational numbers then you need to use a Rational class. float and double won't do it, and neither will BigDecimal. – President James K. Polk Nov 24 '17 at 00:07

0 Answers0