0

Eeveryone try to run this code,

Public Class Form1

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load
        MsgBox(968 * 0.05 - 55)
    End Sub
End Class

the result is: -6.5999999999999 Why is not -6.6? thers is problem with VS?

Nguyen Duc Linh
  • 81
  • 1
  • 7

2 Answers2

1

This is NOT a bug: it is the normal behavior of floating point types.

What you get is the closer approximation of the value you expected. Floating point values are stored using bits (as any other value type) but not all the numbers are exactly representable by bits.

If you are willing to display any floating point value to the user (like through MsgBox as you did), then you should format it, like (as a very siple example):

MsgBox((968 * 0.05 - 55).ToString("F2"))

that keeps only the first 2 decimal digits (if needed) and approximate the internal (binary) value to that textual representation.

Regards,
Daniele.

Rubidium 37
  • 661
  • 6
  • 12
1

No the problem isnt in Visual Studio. It is because the algorithm to calculate.

http://floating-point-gui.de/basic/

Because internally, computers use a format (binary floating-point) that cannot accurately represent a number like 0.1, 0.2 or 0.3 at all.

When the code is compiled or interpreted, your “0.1” is already rounded to the nearest number in that format, which results in a small rounding error even before the calculation happens. Why do computers use such a stupid system?

It’s not stupid, just different. Decimal numbers cannot accurately represent a number like 1/3, so you have to round to something like 0.33 - and you don’t expect 0.33 + 0.33 + 0.33 to add up to 1, either - do you?

Computers use binary numbers because they’re faster at dealing with those, and because for most calculations, a tiny error in the 17th decimal place doesn’t matter at all since the numbers you work with aren’t round (or that precise) anyway.

AngularLover
  • 364
  • 1
  • 12