1

Multiplying an int with a float will result in an incorrect value using g++ / visual studio

It will always work fine when using double instead of float.

The calculation is also very simple - 1000 * 0.01 - should be 10 - but its 9.

Sample Program

#include <iostream>
#include <stdio.h>

typedef unsigned long long int64;
typedef unsigned int       int32;

int main()
{
    int64 test = 1000 * 0.01f;
    printf("Test %u\n",test); // Prints 10

    int64 test2 = 1000;
    test2 = test2 * 0.01f;
    printf("Test2 %u\n",test2); // Prints 9

    int32 test3 = 1000;

    test3 = test3 * 0.01f;
    printf("Test3 %i\n",test3); // Prints 9

    int64 test4 = 1000;
    test4 = test4 * 0.01; 
    printf("Test4 %u\n",test4); // Prints 10

    int32 test5 = 1000;

    test5 = test5 * 0.01;
    printf("Test5 %i\n",test5); // Prints 10

}

output

Test 10
Test2 9
Test3 9
Test4 10
Test5 10

Compiled on Wheezy 32bit using g++ main.cpp -o main

EDIT: This was just an oversimplification!!!

Basically i do NOT use 0.01f

What i do is this convert one unit to another, e.g. one of my units is 1:100 to another unit. Another one is 1:10 or 1:1000.

So i basically calculate unit1/unit2 to have the correct multiplier if i want to sum 2 different units.

In that case it was unit1 = 1000 unit2 = 10

so 10/1000 = 0.01 * 1000 = 10

So when i want to convert 1000 pieces of unit1 into unit2 it should result in 10 because basically unit1 is 1000 of base, and unit2 is 10 base.

Steve
  • 738
  • 1
  • 9
  • 30
  • 4
    [What every computer programmer should know about floating point](http://blog.reverberate.org/2014/09/what-every-computer-programmer-should.html) – Mooing Duck Jan 20 '15 at 22:36
  • Welcome to floating point arithmetic. That's because the expression evaluates to something like 9.9999991598134 (making up digits here), so when you truncate it, that's 9. – Barry Jan 20 '15 at 22:37
  • 1
    When you type `0.01f`, the computer sees `0.00999999977648258209228515625` http://coliru.stacked-crooked.com/a/1c0d5bd0b0272d7d – Mooing Duck Jan 20 '15 at 22:38
  • If you try writing down 0.01 in binary, you'll find out that it is periodic. So with a finite number of digits, you'll have to truncate which might mean to round down. Then 1000 (0.01 - ɛ) < 10. What happens to 1 / 10 in binary is just the same that happens to 1 / 3 in decimal. – 5gon12eder Jan 20 '15 at 22:39
  • The title of the dupe is hilarious. Do we also have “Is `select(2)` broken?” – 5gon12eder Jan 20 '15 at 22:44
  • edited my question - i oversimplified the issue - i do not use 0.01f – Steve Jan 20 '15 at 22:51
  • You can use http://babbage.cs.qc.cuny.edu/IEEE-754.old/Decimal.html to see what your numbers actually are when stored as a floating point. It doesn't matter if it's 0.01f. There's lots of other numbers that can't be represented as a float or a double – Alex Jan 21 '15 at 01:30

1 Answers1

2

Welcome to the world of floating point numbers. What is happening here is that 0.01f is a repeating decimal in binary, so 1000 * 0.01f would equal 9.99999999... to a certain amount of places I can't remember. Afterwards, there is integer truncation, so 9.9999... = 9.

Why is 0.01f a repeating decimal? Here is the anatomy of a floating point. http://www.johndcook.com/blog/2009/04/06/anatomy-of-a-floating-point-number/

For more information on the subject, Computerphile has a interesting video.

http://www.youtube.com/watch?v=PZRI1IfStY0

Orange Mushroom
  • 431
  • 4
  • 16
  • I edited my question - i oversimplified the issue - i do not use 0.01f – Steve Jan 20 '15 at 22:50
  • "Why is 0.01f a repeating decimal?" Simpler answer: because 0.01 is 1/100, that is 1/(2^2 * 5^2), and in base 2 any division by 5 is periodic. – Narcolessico Aug 03 '17 at 07:39