2

I receive very interesting and seemingly erratic results in irb for Ruby. What is going on?

This is correct!

>> 23+9.22 
=> 32.22

This is not!

>> 23+9.23 
=> 32.230000000000004

Where are all the trailing zeroes coming from? What is going on?

Andy Shin
  • 74
  • 8

2 Answers2

9
puts "%.30f" % 9.23
#=> 9.230000000000000426325641456060

Because of the difference between base-2 (computer internal representation of numbers) and base-10 (what you are used to dealing with and are typing into your text editor or IRB), you cannot represent 9.23 exactly as a floating point value. This is not specific to Ruby, but present in nearly every programming language.

For example, 9.23 is represented internally as a sum of powers of two, something like:

01001.00111010111...
||||| ||||||||||+-> 1 * 1/2048 \            
||||| |||||||||+--> 1 * 1/1024  \          
||||| ||||||||+---> 1 * 1/512    \        
||||| |||||||+----> 0 * 1/256     \       
||||| ||||||+-----> 1 * 1/128      \      
||||| |||||+------> 0 * 1/64         fraction = 0.22998046875
||||| ||||+-------> 1 * 1/32       /     
||||| |||+--------> 1 * 1/16      /      
||||| ||+---------> 1 * 1/8      /      
||||| |+----------> 0 * 1/4     /       
||||| +-----------> 0 * 1/2    /        
|||||  
||||+-------------> 1 * 1      \
|||+--------------> 0 * 2       \  
||+---------------> 0 * 4         integer = 9
|+----------------> 1 * 8       /
+-----------------> 0 * 16     /

For more information, go read What Every Computer Scientist Should Know About Floating Point Arithmetic and the IEEE 754 description on Wikipedia.

Your workarounds, when precision is important (such as currency) are to:

  1. Keep track of the smallest available unit for your calculations (e.g. pennies) and only convert to decimal for display. For example:

    result = 2300 + 923          # $23.00 + $9.23
    puts "%.2f" % (result/100.0) # Ensure output rounded to two decimal places
    #=> 32.23
    
  2. Use a library like BigDecimal to keep track of exact precision for you, and set your database to use a fixed-precision data type like DECIMAL or NUMERIC.

  3. Accept that the numeric values will usually be "close enough", and always use a formatting option when displaying them to the user, e.g.

    result = 23 + 9.27    #=> 32.269999999999996
    puts "%g" % result    #=> 32.27
    puts "%.3f" % result  #=> 32.270
    puts "%.1f" % result  #=> 32.3
    
Phrogz
  • 296,393
  • 112
  • 651
  • 745
  • 1
    There should be a hat awarded on SO whenever someone uses a link to WECSSKAFPA. I have personally pointed people at it about 100 times. I can always use moar hatz. – Warren P Oct 07 '16 at 13:08
0

This is not a problem with Ruby. This is a know problem of representation of floating point numbers in computers.

If you are doing financial or monetary processing, then you want to stay away from floating point numbers to represent your amounts.

Gerard Yin
  • 1,283
  • 1
  • 12
  • 24