Why does a division result differ based on the cast type?

Question

Here's a part of code that I dont understand:

byte b1 = (byte)(64 / 0.8f); // b1 is 79
int b2 = (int)(64 / 0.8f); // b2 is 79
float fl = (64 / 0.8f); // fl is 80

Why are the first two calculations off by one? How should I perform this operation, so its fast and correct?

EDIT: I would need the result in byte

Answer 1

I'm afraid fast and correct are at odds in cases like this.

Binary floating point arithmetic almost always creates small errors, due to the underlying representation in our CPU architectures. So in your initial expression you actually get a value a little bit smaller than the mathematically correct one. If you expect an integer as the result of a particular mathematic operation and you get something very close to it, you can use the Math.Round(Double, MidpointRounding) method to perform the correct rounding and compensate for small errors (and make sure you pick the MidpointRounding strategy you expect).

Simply casting the result to a type such as byte or int doesn't do rounding - it simply cuts off the fractional part (even 1.99999f will become 1 when you just cast it to these types).

Decimal floating point arithmetic is slower and more memory intensive, but doesn't cause these errors. To perform it, use decimal literals instead of float literals (e.g. 64 / 0.8m).

The rule of thumb is:

• If you are dealing with exact quantities (typically man-made, like money), use decimal.
• If you are dealing with inexact quantities (like fractional physical constants or irrational numbers like π), use double.
• If you are dealing with inexact quantities (as above) and some accuracy can be further sacrificed for speed (like when working with graphics), use float.

Answer 2

EDIT: Not entirely correct, see: Why does a division result differ based on the cast type? (Followup)

Rounding issue: By converting to byte / int, you are clipping of the decimal places.

But 64 / 0.8 should not result in any decimal places? Wrong: Due to the nature of floating point numbers, 0.8f can not be represented exactly like that in memory; it is stored as something close to 0.8f (but not exactly). See Floating point inaccuracy examples or similar threads. Thus, the result of the calculation is not 80.0f, but 79.xxx where xxx is close to 1 but still not exactly one.

You can verify this by typing the following into the Immediate Window in Visual Studio:

(64 / 0.8f)
80.0
(64 / 0.8f) - 80
-0.0000011920929
100 * 0.8f - 80
0.0000011920929

You can solve this by using rounding:

byte b1 = (byte)(64 / 0.8f + 0.5f);
int b2 = (int)(64 / 0.8f + 0.5f);
float fl = (64 / 0.8f);

Answer 3

To understand the problem, you need to understand the basics of floating point representation and operations.

0.8f can not be exactly represented in memory using a floating point number.

In mathematics, 64/0.8 equals 80. In floating point arithmetics, 60/0.8 equals approximatively 80.

When you cast a float to an integer or a byte, only the integer part of the number is kept. In your case, the imprecise result of the floating point division is a little bit smaller than 80 hence the conversion to integer yields 79.

If you need an integer result, I would suggest you to round the result instead of casting it. One way to do it is to use the following function, that convert to an integer by rounding to the closest integer :

Convert.ToInt32(64/0.8f);