Dividing using 10 gives wrong precision

Question

My intention is to round a value but it gives wrong output

      float travellingEmptyDist = 2.05; 
travellingEmptyDist = round(travellingEmptyDist * 10); // gives value 21.0
travellingEmptyDist = travellingEmptyDist/ 10.0; // gives wrong value 2.09999, expecting 2.1

Is there any way to rectify it

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html — phuclv, Apr 03 '14 at 17:28
Someone should at least point out that `travellingEmptyDist` is _not_ 2.05, despite what his source code says. — James Kanze, Apr 03 '14 at 17:30
related question and excellent answer: https://stackoverflow.com/questions/12635537/c-floating-point-precision — JSQuareD, Apr 03 '14 at 17:52

David Zech · Answer 1 · 2014-04-03T17:52:47.163

1

I'd suggest watching this video Computerphile: Floating Points

It'll give you a very simple overview about why floating points inherently are imprecise without going into too much technical detail. I'm also bias because I love numberphile.

You need to ask yourself, do you really need that extra 0.00001 of precision? If you really need more precision, use a double instead of a float, but even then you'll still have rounding errors.

If you need even higher precision, you're going to have to use a 3rd party precision arithmetic library. I've never had the need for one, but GMP seems to be popular based on Google results.

edited Apr 03 '14 at 17:52

answered Apr 03 '14 at 17:42

David Zech

745
3
14

Double will get you that 0.00001f, but it won't get you "true" percision. For that you need to do fixed point math: http://stackoverflow.com/questions/79677/whats-the-best-way-to-do-fixed-point-math – IdeaHat Apr 03 '14 at 17:49
That is why I linked to GMP – David Zech Apr 03 '14 at 17:51

Dividing using 10 gives wrong precision

1 Answers1