Compare double in VBA precision problem

Question

I have trouble comparing 2 double in Excel VBA

suppose that I have the following code

Dim a as double
Dim b as double
a = 0.15
b = 0.01

After a few manipulations on b, b is now equal to 0.6

however the imprecision related to the double data type gives me headache because

if a = b then
 //this will never trigger
end if

Do you know how I can remove the trailing imprecision on the double type?

score 16 · Accepted Answer · edited May 05 '16 at 23:21

16

You can't compare floating point values for equality. See this article on "Comparing floating point numbers" for a discussion of how to handle the intrinsic error.

It isn't as simple as comparing to a constant error margin unless you know for sure what the absolute range of the floats is beforehand.

edited May 05 '16 at 23:21

DHW

1,176
1
15
30

answered Oct 24 '08 at 22:12

Rob Walker

46,588
15
99
136

2

For another reference you can check out http://msdn.microsoft.com/en-us/library/ae382yt8(VS.80).aspx – ZCHudson Oct 24 '08 at 22:14
I wrote some code to address the problems in that linked article: https://stackoverflow.com/a/68291403/6609896 – Greedo Jul 07 '21 at 18:30

score 8 · Answer 2 · answered Apr 07 '11 at 06:14

if you are going to do this....

Dim a as double  
 Dim b as double  
 a = 0.15  
 b = 0.01

you need to add the round function in your IF statement like this...

  If Round(a,2) = Round(b,2) Then   
     //code inside block will now trigger.
  End If

See also here for additional Microsoft reference.

score 5 · Answer 3 · answered Oct 24 '08 at 22:09

It is never wise to compare doubles on equality.

Some decimal values map to several floating point representations. So one 0.6 is not always equal to the other 0.6.

If we subtract one from the other, we probably get something like 0.00000000051.

We can now define equality as having a difference smaller that a certain error margin.

score 5 · Answer 4 · answered Jun 04 '14 at 15:14

Here is a simple function I wrote:

Function dblCheckTheSame(number1 As Double, number2 As Double, Optional Digits As Integer = 12) As Boolean

If (number1 - number2) ^ 2 < (10 ^ -Digits) ^ 2 Then
    dblCheckTheSame = True
Else
    dblCheckTheSame = False
End If

End Function

Call it with:

MsgBox dblCheckTheSame(1.2345, 1.23456789)
MsgBox dblCheckTheSame(1.2345, 1.23456789, 4)
MsgBox dblCheckTheSame(1.2345678900001, 1.2345678900002)
MsgBox dblCheckTheSame(1.2345678900001, 1.2345678900002, 14)

Greedo · Answer 5 · 2023-08-07T09:49:05.567

Late answer, but I'm surprised a solution hasn't been posted yet that addresses the concerns outlined in the article linked in the (currently) accepted answer, namely that:

Rounding checks equality with absolute tolerance (e.g. 0.0001 units if rounded to 4d.p.) which is rubbish when comparing different values on multiple orders of magnitude (so not just comparing to 0)
Relative tolerance that scales with one of the numbers being compared meanwhile is not mentioned in the current answers, but performs well on non-zero comparisons (however will be bad at comparing to zero as the scaling blows up around then).

To solve this, I've taken inspiration from Python: PEP 485 -- A Function for testing approximate equality to implement the following (in a standard module):

Code

'@NoIndent: Don't want to lose our description annotations
'@Folder("Tests.Utils")

Option Explicit
Option Private Module

'Based on Python's math.isclose https://github.com/python/cpython/blob/17f94e28882e1e2b331ace93f42e8615383dee59/Modules/mathmodule.c#L2962-L3003
'math.isclose -> boolean
'    a: double
'    b: double
'    relTol: double = 1e-09
'        maximum difference for being considered "close", relative to the
'        magnitude of the input values, e.g. abs(a - b)/(a OR b) < relTol
'    absTol: double = 0.0
'        maximum difference for being considered "close", regardless of the
'        magnitude of the input values, e.g. abs(a - b) < absTol
'Determine whether two floating point numbers are close in value.
'Return True if a is close in value to b, and False otherwise.
'For the values to be considered close, the difference between them
'must be smaller than at least one of the tolerances.
'-inf, inf and NaN behave similarly to the IEEE 754 Standard.  That
'is, NaN is not close to anything, even itself.  inf and -inf are
'only close to themselves.
'@Description("Determine whether two floating point numbers are close in value, accounting for special values in IEEE 754")
Public Function IsClose(ByVal a As Double, ByVal b As Double, _
                        Optional ByVal relTol As Double = 0.000000001, _
                        Optional ByVal absTol As Double = 0 _
                        ) As Boolean
                        
    If relTol < 0# Or absTol < 0# Then
        Err.Raise 5, Description:="tolerances must be non-negative"
    ElseIf a = b Then
        'Short circuit exact equality -- needed to catch two infinities of
        ' the same sign. And perhaps speeds things up a bit sometimes.
        IsClose = True
    ElseIf IsInfinity(a) Or IsInfinity(b) Then
        'This catches the case of two infinities of opposite sign, or
        ' one infinity and one finite number. Two infinities of opposite
        ' sign would otherwise have an infinite relative tolerance.
        'Two infinities of the same sign are caught by the equality check
        ' above.
        IsClose = False
    Else
        'Now do the regular computation on finite arguments. Here an
        ' infinite tolerance will always result in the function returning True,
        ' since an infinite difference will be <= to the infinite tolerance.
        'NaN has already been filtered out in the equality checks earlier.

        On Error Resume Next 'This is to suppress overflow errors as we deal with infinity.
        Dim diff As Double: diff = Abs(b - a)
        
        If diff <= absTol Then
            IsClose = True
        ElseIf diff <= CDbl(Abs(relTol * b)) Then
            IsClose = True
        ElseIf diff <= CDbl(Abs(relTol * a)) Then
            IsClose = True
        End If
        On Error GoTo 0
    End If
End Function

'@Description "Checks if Number is IEEE754 +/- inf, won't raise an error"
Public Function IsInfinity(ByVal Number As Double) As Boolean
    On Error Resume Next                         'in case of NaN
    IsInfinity = Abs(Number) = PosInf
    On Error GoTo 0
End Function

'@Description "IEEE754 -inf"
Public Property Get NegInf() As Double
    On Error Resume Next
    NegInf = -1 / 0
    On Error GoTo 0
End Property

'@Description "IEEE754 +inf"
Public Property Get PosInf() As Double
    On Error Resume Next
    PosInf = 1 / 0
    On Error GoTo 0
End Property

'@Description "IEEE754 signaling NaN (sNaN)"
Public Property Get NaN() As Double
    On Error Resume Next
    NaN = 0 / 0
    On Error GoTo 0
End Property

'@Description "IEEE754 quiet NaN (qNaN)"
Public Property Get QNaN() As Double
    QNaN = -NaN
End Property

_{Updated to incorporate great feedback on Code Review from Cristian Buse}

Examples

The IsClose function can be used to check for absolute difference:

assert(IsClose(0, 0.0001233, absTol:= 0.001)) 'same to 3 d.p.?

... or relative difference:

assert(IsClose(1234.5, 1234.6, relTol:= 0.0001)) '0.01% relative difference?

... but generally you specify both and if either tolerance is met then the numbers are considered close. It has special handling of +-infinity which are only close to themselves, and NaN which is close to nothing (see the PEP for full justification, or my Code Review post where I'd love feedback on this code :)

score 2 · Answer 6 · answered Oct 24 '08 at 23:18

2

As has been pointed out, many decimal numbers cannot be represented precisely as traditional floating-point types. Depending on the nature of your problem space, you may be better off using the Decimal VBA type which can represent decimal numbers (base 10) with perfect precision up to a certain decimal point. This is often done for representing money for example where 2-digit decimal precision is often desired.

Dim a as Decimal
Dim b as Decimal
a = 0.15
b = 0.01

answered Oct 24 '08 at 23:18

C. Dragon 76

9,882
9
34
41

2

Is decimal available in vba? I think you need to declare as variant then cast to decimal via Cdec(myVar). – Mark Nold Oct 25 '08 at 16:26
We are talking about VBA. Decimal is not an VariableType there. So this code does not compile. – Gener4tor Nov 05 '20 at 11:15

score 1 · Answer 7 · answered Oct 27 '08 at 21:36

1

The Currency data type may be a good alternative. It handles relatively large numbers with fixed four digit precision.

answered Oct 27 '08 at 21:36

DJ.

16,045
3
42
46

score 1 · Answer 8 · answered Jul 05 '22 at 06:34

Depending on your situation and your data, and if you're happy with the level of precision shown by default, you can try comparing the string conversions of the numbers as a very simple coding solution:

if cstr(a) = cstr(b)

This will include as much precision as would be displayed by default, which is generally sufficient to consider the numbers equal.

This would be inefficient for very large data sets, but for me was useful when reconciling imported data which was identical but was not matching after storing the data in VBA Arrays.

Josh Anstead · Answer 9 · 2019-06-06T19:34:37.940

Work-a-round?? Not sure if this will answer all scenarios, but I ran into a problem comparing rounded double values in VBA. When I compared to numbers that appeared to be identical after rounding, VBA would trigger false in an if-then compare statement. My fix was to run two conversions, first double to string, then string to double, and then do the compare.

Simulated Example I did not record the exact numbers that caused the error mentioned in this post, and the amounts in my example do not trigger the problem currently and are intended to represent the type of issue.

 Sub Test_Rounded_Numbers()

      Dim Num1 As Double

      Dim Num2 As Double

      Let Num1 = 123.123456789

      Let Num2 = 123.123467891

      Let Num1 = Round(Num1, 4) '123.1235


      Let Num2 = Round(Num2, 4) '123.1235

      If Num1 = Num2 Then

           MsgBox "Correct Match, " & Num1 & " does equal " & Num2
      Else
           MsgBox "Inccorrect Match, " & Num1 & " does not equal " & Num2
      End If

      'Here it would say that "Inccorrect Match, 123.1235 does not equal 123.1235."

 End Sub

 Sub Fixed_Double_Value_Type_Compare_Issue()

      Dim Num1 As Double

      Dim Num2 As Double

      Let Num1 = 123.123456789

      Let Num2 = 123.123467891

      Let Num1 = Round(Num1, 4) '123.1235


      Let Num2 = Round(Num2, 4) '123.1235

      'Add CDbl(CStr(Double_Value))
      'By doing this step the numbers
      'would trigger if they matched
      '100% of the time

      If CDbl(CStr(Num1)) = CDbl(CStr(Num2)) Then

           MsgBox "Correct Match"
      Else
           MsgBox "Inccorrect Match"

      End If

      'Now it says Here it would say that "Correct Match, 123.1235 does equal 123.1235."
 End Sub

Elimar · Answer 10 · 2020-05-24T15:15:34.287

-1

Try to use Single values if possible. Conversion to Double values generates random errors.

Public Sub Test()
Dim D01 As Double
Dim D02 As Double
Dim S01 As Single
Dim S02 As Single
S01 = 45.678 / 12
S02 = 45.678
D01 = S01
D02 = S02
Debug.Print S01 * 12
Debug.Print S02
Debug.Print D01 * 12
Debug.Print D02
End Sub

    45,678 
    45,678 
    45,67799949646 
    45,6780014038086

edited May 24 '20 at 15:15

answered May 23 '20 at 12:14

Elimar

9
1

The wording of this is a bit misleading, the errors aren't random, but the solution is good as a single will round to a reasonable number of significant figures for comparison. Still better I think to control the rounding using one of the other approaches – Greedo Jul 10 '21 at 09:06

Compare double in VBA precision problem

10 Answers10

Code

Examples

Linked

Related