0

I'm working through C++ Primer Plus, using Xcode as my IDE and one of the exercises calls for assigning the value 0.0254 to a symbolic constant (converting inches to meters). The problem is, when I declare the constant I'm getting a value of 0.0253999997. I'm declaring the constant as seen below.

#include <iostream>
const float METERS_PER_INCH = .0254;
Cœur
  • 37,241
  • 25
  • 195
  • 267
Greg Pastorelli
  • 3
  • 1
  • Welcome to Stack Overflow! If you're starting with C++, I recommend [a good introductory C++ book](http://stackoverflow.com/q/388242/46642). Now let me see if I can answer your question :) – R. Martinho Fernandes Dec 03 '11 at 03:57
  • I'll try 0.0254f, thanks. Also, this exercise I'm having the issue with is from the C++ Primer, Chapter 3, exercise #2 (converting height in inches to meters). – Greg Pastorelli Dec 03 '11 at 05:13
  • possible duplicate of [strange output in comparision of float with float literal](http://stackoverflow.com/questions/1839422/strange-output-in-comparision-of-float-with-float-literal) – dmckee --- ex-moderator kitten Dec 03 '11 at 20:20

2 Answers2

1

This is the issue of rounding error due to floats storing numbers in a base 2 number system(think about how we cannot write 1/3 without rounding in our base 10 number system). It results in small rounding errors like the ones you are seeing when you store non-base 2 numbers.

The solution is to use integers or a bignum library(I suggest the GNU Multiple Precision library ). A bignumber library uses integers to store arbitrary precision numbers exactly.

0

Don't worry your mac is OK.

Problem is that computers can't accurately represent floats and doubles so this is what you are seeing there.

Why comparing double and float leads to unexpected result?

http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

strange output in comparison of float with float literal

Google is your friend!

Community
  • 1
  • 1
FailedDev
  • 26,680
  • 9
  • 53
  • 73