-1

I am taking an input from a textbox from a user. That value is converted to a float using the following line:

float x = float.Parse(txt1.Text) * 8;

The float variable is then converted to an integer using the following line:

int switchOnDelayValue = (int)Math.Round(x);

Then I convert the integer into a hexadecimal equivalent string using:

valueToString = Convert.ToString(switchOnDelayValue, 16).PadLeft(8, '0');

The conversion from float to int works fine for mostly all values, except for some values near the maximum value of int. For example, when I input the value 268,435,455

float x = float.Parse(txt1.Text) * 8

x should now store 2,147,483,640. However, the float stores the value as 2.14748365E+09. And when the float is casted as an int, it overflows and gives a negative value.

Uwe Keim
  • 39,551
  • 56
  • 175
  • 291
Muhammad Ali
  • 277
  • 3
  • 11
  • 5
    A [float](https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/builtin-types/floating-point-numeric-types) has a precision of around 6-9 digits. You are experiencing the drawbacks of using a `float` when you want a more precise numeric type. You could consider a `double` which has a precision of ~15-17 digits and you *may* not run into the same issue. – JNevill Nov 14 '19 at 14:22
  • It's a rounding error caused by the limitations of a 32-bit floating point number. A float can only hold up to 9 integer digits accurately. See https://en.wikipedia.org/wiki/Single-precision_floating-point_format – Matthew Watson Nov 14 '19 at 14:22

1 Answers1

1

From the docs, float only has a precision of ~6-9 digits.

You're seeing a difference around the 9th digit. This is not unexpected. You probably need a double, which has a precision of ~15-17 digits, or a decimal which has a precision of 28-39 digits (but lower range).

canton7
  • 37,633
  • 3
  • 64
  • 77