15

The machine epsilon is canonically defined as the smallest number which added to one, gives a result different from one.

There is a Double.Epsilon but the name is very misleading: it is the smallest (denormalized) Double value representable, and thus useless for any kind of numeric programming.

I'd like to get the true epsilon for the Double type, so that not to have to hardcode tolerances into my program. How do I do this ?

Alexandre C.
  • 55,948
  • 11
  • 128
  • 197
  • Relevant: http://www.johndcook.com/blog/2010/06/08/c-math-gotchas/ – AakashM Feb 22 '12 at 10:28
  • @AakashM: I read that. The meaning of epsilon is rather clear in view of IEEE754, and it is a shame that Microsoft did something this amateurish. Is their floating point implementation to be trusted ? – Alexandre C. Feb 22 '12 at 10:36
  • @AlexandreC.: `Double.MinValue` is most likely defined as it is to correspond to the other `MinValue` fields in the .NET Framework like `Int32.MinValue`, `DateTime.MinValue` etc. This obviously is not the same as `DBL_MIN` in C. However, I agree that the definition of `Double.Epsilon` is confusing. – Martin Liversage Feb 22 '12 at 10:54

5 Answers5

9

It's(on my machine):

   1.11022302462516E-16

You can easily calculate it:

        double machEps = 1.0d;

        do {
           machEps /= 2.0d;
        }
        while ((double)(1.0 + machEps) != 1.0);

        Console.WriteLine( "Calculated machine epsilon: " + machEps );

Edited:

I calcualted 2 times epsilon, now it should be correct.

Meonester
  • 220
  • 1
  • 3
  • @Meonester Do you mean to divide machEps by 4 each time (once in the body and once in the condition? If I leave out the condition division ( i.e. while ((double)(1.0 + (machEps)) != 1.0); ) I get a value for machEps of 1.11022302462516E-16. – AlanT Feb 22 '12 at 10:43
  • @AlanT Right, which matches what the `Math.NET` code suggests it is. – Christian.K Feb 22 '12 at 11:01
  • Yes, you are right: Epsilon = 1.11022302462516E-16. There should be: while ((double)(1.0 + (machEps)) != 1.0); – Meonester Feb 22 '12 at 11:11
  • @Meonester Unless you're already in the process of doing it, you should then edit your answer accordingly. – Christian.K Feb 22 '12 at 11:13
  • A small nitpick/curiosity: This approach gives the correct answer from the perspective of the binary number itself, but if we're talking about the decimal literal then there's a smaller number that will produce the same binary representation. So, for example, on my machine the smallest *decimal* representation of the epsilon is actually `1.1102230246251566639E-16`. (Of course, `1.11022302462516E-16` and `1.1102230246251566639E-16` have *exactly* the same binary representation so are to-all-intents-and-purposes the same number.) – LukeH Feb 22 '12 at 11:59
  • 2
    Wasn't it right the first time? When the loop terminates `(1.0 + machEps) == 1.0` which isn't what we want. We want the last value such that `(1.0 + machEps) != 1.0`. – Rup Feb 22 '12 at 17:11
  • After the last iteration it has to be doubled again so that machEps + 1 is actually different than 1. `machEps *= 2.0d;` should be added after the loop. And the result should be `2.22044604925031E-16`. – Falk Aug 29 '19 at 07:05
8

The Math.NET library defines a Precision class, which has a DoubleMachineEpsilon property.

You could check how they do it.

According to that it is:

    /// <summary>
    /// The base number for binary values
    /// </summary>
    private const int BinaryBaseNumber = 2;

    /// <summary>
    /// The number of binary digits used to represent the binary number for a double precision floating
    /// point value. i.e. there are this many digits used to represent the
    /// actual number, where in a number as: 0.134556 * 10^5 the digits are 0.134556 and the exponent is 5.
    /// </summary>
    private const int DoublePrecision = 53;

    private static readonly double doubleMachinePrecision = Math.Pow(BinaryBaseNumber, -DoublePrecision);

So it is 1,11022302462516E-16 according to this source.

Christian.K
  • 47,778
  • 10
  • 99
  • 143
  • It is quite complex bit fiddling, needing to know much more about the internal representation of `System.Double` that I am ready to tackle (it is not *just* IEEE754 stuff, but also endianness issues, etc). – Alexandre C. Feb 22 '12 at 10:33
  • @Alexandre: I'm pretty sure that the internal binary representation is mandated by IEEE-754 itself. There's a good description of the internals of `double` [here](http://csharpindepth.com/Articles/General/FloatingPoint.aspx). There's also a link to Jon's [DoubleConverter.cs](http://pobox.com/~skeet/csharp/DoubleConverter.cs) class in there too; looking at that might offer some clues. – LukeH Feb 22 '12 at 11:08
  • @LukeH: Now that I notice that Microsoft cannot get even the most basic terminology right, I cannot assume that their floating point implementation is correct for all purposes. – Alexandre C. Feb 22 '12 at 12:26
  • 1
    The majority of the floating point implementation is in hardware, though, so I doubt there's much to worry about. In any case nowhere on the documentation page for `Double.Epsilon` does it say "machine epsilon" which is the correct term for what you want, so I'm not sure it's that confusing. – Rup Feb 22 '12 at 17:15
  • 2
    @Alexandre, @Rup: ...and IEEE-754 doesn't define "machine epsilon" at all, so the fact that `double.Epsilon` and machine epsilon aren't the same thing seems completely orthogonal to whether or not Microsoft's FP implementation is correct. – LukeH Feb 22 '12 at 18:34
2

Just hard-code the value:

const double e1 = 2.2204460492503131e-16;

or use the power of two:

static readonly double e2 = Math.Pow(2, -52);

or use your definition (more or less):

static readonly double e3 = BitConverter.Int64BitsToDouble(BitConverter.DoubleToInt64Bits(1.0) + 1L) - 1.0;

And see Wikipedia: machine epsilon.

Jeppe Stig Nielsen
  • 60,409
  • 11
  • 110
  • 181
0

LAPACK + DLAMCH, 64-bit INTEL processor, C#:

var pp = double.Epsilon; // pp = 4.94065645841247E-324
double p = NativeMethods.MachinePrecision('S'); // =DLAMCH('S') 
p = 2.2250738585072014E-308
double.MinValue = -1.7976931348623157E+308
double.MaxValue =  1.7976931348623157E+308
Erik
  • 894
  • 1
  • 8
  • 25
-2

Ref. the routine in Meonester's: Actually the value of machEps on exit from the do ... while loop is such that 1+machEps == 1. To obtain the machine epsilon we must go back to the previous value, by adding the following after the loop: machEps *= 2.0D; This will return 2.2204460492503131e-16 in agreement with the recommendation in Microsoft's documentation for Double.Epsilon.

GiorgioITA
  • 1