Arithmetic calculation using Linq.Expressions yields different results on 32 vs 64bit

Question

I am observing some strange behavior regarding the results of the following code:

namespace Test {
  class Program {
    private static readonly MethodInfo Tan = typeof(Math).GetMethod("Tan", new[] { typeof(double) });
    private static readonly MethodInfo Log = typeof(Math).GetMethod("Log", new[] { typeof(double) });

    static void Main(string[] args) {
    var c1 = 9.97601998143507984195821336470544338226318359375d;
    var c2 = -0.11209109500765944422706610339446342550218105316162109375d;

    var result1 = Math.Pow(Math.Tan(Math.Log(c1) / Math.Tan(c2)), 2);

    var p1 = Expression.Parameter(typeof(double));
    var p2 = Expression.Parameter(typeof(double));
    var expr = Expression.Power(Expression.Call(Tan, Expression.Divide(Expression.Call(Log, p1), Expression.Call(Tan, p2))), Expression.Constant(2d));
    var lambda = Expression.Lambda<Func<double, double, double>>(expr, p1, p2);
    var result2 = lambda.Compile()(c1, c2);

    var s1 = DoubleConverter.ToExactString(result1);
    var s2 = DoubleConverter.ToExactString(result2);

    Console.WriteLine("Result1: {0}", s1);
    Console.WriteLine("Result2: {0}", s2);
  }
}

The code compiled for x64 gives the same result:

Result1: 4888.95508254035303252749145030975341796875
Result2: 4888.95508254035303252749145030975341796875

But when compiled for x86 or Any Cpu, the results differ:

Result1: 4888.95508254035303252749145030975341796875
Result2: 4888.955082542781383381225168704986572265625

Why does result1 stay the same while result2 depends on the target architecture? Is there any way to make result1 and result2 stay the same on the same architecture?

The DoubleConverter class is taken from http://jonskeet.uk/csharp/DoubleConverter.cs. Before you tell me to use decimal I don't need more precision, I just need the results to be consistent. The target framework is .NET 4.5.2 and the test project was built in debug mode. I am using Visual Studio 2015 Update 1 RC on Windows 10.

Thanks.

EDIT

At user djcouchycouch's suggestion I tried to further simplify the example:

  var c1 = 9.97601998143507984195821336470544338226318359375d;
  var c2 = -0.11209109500765944422706610339446342550218105316162109375d;
  var result1 = Math.Log(c1) / Math.Tan(c2);
  var p1 = Expression.Parameter(typeof(double));
  var p2 = Expression.Parameter(typeof(double));
  var expr = Expression.Divide(Expression.Call(Log, p1), Expression.Call(Tan, p2));
  var lambda = Expression.Lambda<Func<double, double, double>>(expr, p1, p2);
  var result2 = lambda.Compile()(c1, c2);

x86 or AnyCpu, Debug:

Result1: -20.43465311535924655572671326808631420135498046875
Result2: -20.434653115359243003013034467585384845733642578125

x64, Debug:

Result1: -20.43465311535924655572671326808631420135498046875
Result2: -20.43465311535924655572671326808631420135498046875

x86 or AnyCpu, Release:

Result1: -20.434653115359243003013034467585384845733642578125
Result2: -20.434653115359243003013034467585384845733642578125

x64, Release:

Result1: -20.43465311535924655572671326808631420135498046875
Result2: -20.43465311535924655572671326808631420135498046875

The point is that results vary between Debug, Release, x86 and x64, and the more complicated the formula, the more likely it would cause bigger deviations.

I cannot reproduce this issue on VS2012 (Windows 7 SP1). Both architectures yield the same result. — Matias Cicero, Nov 20 '15 at 18:30
I see the same result as the OP on Windows 7, VS2015 x86/Any CPU. — Ian P, Nov 20 '15 at 18:36
Reproduced on Windows 7, VS2015. The result depends on configuration and debugger attached. In release mode, the result is different when running with debugger (F5) and without debugger (Ctrl+F5). — Andrey Nasonov, Nov 20 '15 at 18:39
The 32bit JIT uses x87 instructions and 80bit temporary results, the 64bit JIT does not. But with x87 instructions, if it saves a temporary to memory, it gets truncated to 64bit again. So it's sometimes different, but not necessarily, depending on the exact codegen. — harold, Nov 20 '15 at 18:49
Try to reduce the problem to the smallest possible that shows the problem. That'll better pinpoint the place that's causing the difference. — djcouchycouch, Nov 20 '15 at 18:56
Does this require expression trees? Should do the same thing if you write it normally (but not using constants that a compiler can optimize away). — usr, Nov 20 '15 at 19:16
Yes, this is part of a machine learning project where we are evaluating tree-based mathematical expressions over a dataset, and using compiled linq expressions brings a significant speed improvement when the data is large enough. — Bogdan B, Nov 20 '15 at 19:19
No, I mean is it only possible to reproduce it using expression trees? I think they are unrelated. — usr, Nov 20 '15 at 19:26
I think Eric Lippert's response to an [even simpler example warrants attention](http://stackoverflow.com/a/8795656/517852): "The C# compiler, the jitter and the runtime all have broad lattitude to give you more accurate results than are required by the specification, at any time, at a whim -- they are not required to choose to do so consistently and in fact they do not." — Mike Zboray, Nov 20 '15 at 19:32

score 3 · Accepted Answer · answered Nov 20 '15 at 19:23

This is allowed by ECMA-335 I.12.1.3 Handling of floating-point data types:

[...] Storage locations for floating-point numbers (statics, array elements, and fields of classes) are of fixed size. The supported storage sizes are float32 and float64. Everywhere else (on the evaluation stack, as arguments, as return types, and as local variables) floating-point numbers are represented using an internal floating-point type. In each such instance, the nominal type of the variable or expression is either float32 or float64, but its value can be represented internally with additional range and/or precision. [...]

As @harold comments on your question, this allows the 80-bit FPU registers to be used in x86 mode. This is what happens when optimisations are enabled, meaning for your user code, when you build in Release mode and don't debug, but for compiled expressions, always.

In order to make sure you get consistent rounding, you need to store intermediate results in a field or array. This means that in order to reliably get results for your non-Expression version, you need to write it as something like:

var tmp = new double[2];
tmp[0] = Math.Log(c1);
tmp[1] = Math.Tan(c2);
tmp[0] /= tmp[1];
tmp[0] = Math.Tan(tmp[0]);
tmp[0] = Math.Pow(tmp[0], 2);

and then you can safely assign tmp[0] to a local variable.

Yes, it's ugly.

For the Expression version, the actual syntax you need is worse and I won't write it out. It involves Expression.Block to allow multiple unrelated sub-expressions to be executed sequentially, Expression.Assign to assign to array elements or fields, and accesses to those array elements or fields.

When the JIT gets upgraded (if ever) to optimize memory access (which right now it shamefully almost never does) this will break. Or, does array access *force* exact precision by the spec? I think a redundant cast to float/double would also work. — usr, Nov 20 '15 at 19:27
@usr The spec, the part I quoted, says that fields and array elements may not have extra precision. Casts are *not* required to drop excess precision. — , Nov 20 '15 at 19:42
OK, then this works. Could also use a `struct { float F; }` and a helper method channeling through that struct. — usr, Nov 20 '15 at 19:44

Arithmetic calculation using Linq.Expressions yields different results on 32 vs 64bit

1 Answers1