float calculation precision

Question

I've encountered a float calculation precision problem in C#, here is the minimal working example :

int num = 160;
float test = 1.3f;

float result = num * test;
int result_1 = (int)result;
int result_2 = (int)(num * test);
int result_3 = (int)(float)(num * test);

Console.WriteLine("{0} {1} {2} {3}", result, result_1, result_2, result_3);

The code above will output "208 208 207 208", could someone explain something on the weird value of result_2 which should be 208? (binary can not represent 1.3 precisely which will cause float precision problem, but I'm curious on the details)

Cannot reproduce: pasted your code into https://dotnetfiddle.net/ and got `208 208 208 208`. — rollstuhlfahrer, Mar 28 '18 at 06:18
[Related](https://stackoverflow.com/questions/588004/is-floating-point-math-broken). I think there is a dupe somewhere, but there are 43 pages of questions linked to that, so... — user202729, Mar 28 '18 at 06:18
[Reproduced on TIO with Mono](https://tio.run/##bY5NC8IwDIbv/oqwUydz7AO8DE9eFQQPHqXMTgpbC0smSOlvn92HFccCgSTv876kxF2jle77DqV6wvWNJJqirDkiXAwSJ1nCS8sHnLlULDQbmEsqAtU1cIB0nxT@XNWaE5BAGpQ4r4rNQmsFdvWgDu7tiBZ/qRNwTx3C3B5O@yqTfRn2CwtXydyT4xcLg3cctUJdi/jWShInqQQLTGLBpK4z17kNojky8o/6KfNT7kKt7fsP). — user202729, Mar 28 '18 at 06:20
Possible duplicate of [Strange behavior when casting a float to int in C#](https://stackoverflow.com/questions/8911440/strange-behavior-when-casting-a-float-to-int-in-c-sharp) — SᴇM, Mar 28 '18 at 06:22

Slaven Tojić · Accepted Answer · 2018-03-28T07:24:04.260

3

num * test will probably give you a result like 207.9999998... and when you cast this float value to int you get 207, because casting to int will round the result down to the nearest integer in this case 207 (similar as Math.Floor()).

If you assign num * test to a float type like float result = num * test; the value 207.9999998... will be rounded to the nearest float value witch is 208.

Let's summerize:

float result = num * test; gives you 208 because you are assigning num * test to a float type.

int result_1 = (int)result; gives you 208 because you are casting the value of result to int -> (int)208 .

int result_2 = (int)(num * test); gives you 207 because you are casting something like 207.9999998... to int -> (int)207.9999998....

int result_3 = (int)(float)(num * test); gives you 208 because you are first casting 207.9999998... to float which gives you 208 and then you are casting 208 to int.

edited Mar 28 '18 at 07:24

answered Mar 28 '18 at 06:26

Slaven Tojić

2,945
2
14
33

thanks and further asking: 1. you say result will round to float, so CLR will use high precision (like double) to do the mul calculation(IL opcode "mul") ? 2. round to float means round to highest precision that float can represent ? If so I can understand why when I change float to double (above codes), the result output will be 207.999992370605 207 207 207 3. why someone can not reproduce the ouput, depend on your answer, it seems the result will be same (unless it will vary on how CLR convert float to int, like use round, not trunc) – tkokof Mar 28 '18 at 10:32
1

Answer to the first and second question : when calculating 160*1.3f and assigning it to a float type, you are taking the double value 207.99999809265137 and rounding it to the nearest float, which is 208. Regarding the third question I reproduced the same output in Visual Studio so maybe it has to do with the compiler on other IDEs. To be honest I'm not really an expert in this area. I just gave a logical explanation to your question. – Slaven Tojić Mar 28 '18 at 12:48
I test the example codes on some online C# compilers and find only on https://dotnetfiddle.net/ the output will be 208 208 208 208, I doubt its compiler compile "int result_2 = (int)(num * test);" to "int result_2 = (int)(float)(num * test);" etc, I can not check the generated ILs so I can not confirm this ... here is a more detail answer https://stackoverflow.com/questions/8795550/casting-a-result-to-float-in-method-returning-float-changes-result/8795656#8795656 – tkokof Mar 29 '18 at 02:37

SᴇM · Answer 2 · 2018-03-28T10:08:06.253

You can also take a look at C# language specification:

Floating-point operations may be performed with higher precision than the result type of the operation. For example, some hardware architectures support an “extended” or “long double” floating-point type with greater range and precision than the double type, and implicitly perform all floating-point operations using this higher precision type. Only at excessive cost in performance can such hardware architectures be made to perform floating-point operations with less precision, and rather than require an implementation to forfeit both performance and precision, C# allows a higher precision type to be used for all floating-point operations. Other than delivering more precise results, this rarely has any measurable effects. However, in expressions of the form x * y / z, where the multiplication produces a result that is outside the double range, but the subsequent division brings the temporary result back into the double range, the fact that the expression is evaluated in a higher range format may cause a finite result to be produced instead of an infinity.

So basically to answer your question - No, it shouldn't. You can use different types, i.e. decimal or Binary floating points etc. And if you are more interested about floating point concepts and formats, you can read Jeffrey Sax's - Floating Point in .NET part 1: Concepts and Formats.

float calculation precision

2 Answers2