0

Undertaking a code review I have come across some strange behaviour I don't understand in the C# immediate window.

all the variables below are floats and calculate the gradient of a line. Occasionally the line is vertical and so the code does an infinity check and then recomputes like this (don't ask me why, the proper answer is to set m= a large number / float.MaxVal)

m = (endPos.Y - startPos.Y)/(endPos.X + 0.0001f - startPos.X);

this is the output of my debug session in the immediate window:

 >m
 Infinity
 >(endPos.Y - startPos.Y)/(endPos.X + 0.0001f - startPos.X)
 2864375
 >(2249.336f-1962.89844f)/(16899.1445f+0.0001f-16899.1445f)
 Infinity

my question is why the middle statement is giving a different result in the c# immediate window than it gives executing in C# - e.g. why does it NOT equal infinity - I assume that somewhere c# is casting to double even though all variables are double?

I understand that this is on the edge of undefined behaviour for floats as there are only 7 significant figures and we're computing with 9.

what is more strange to me is that one certain servers this statement will never execute to equal infinity and others it will always do so - I assume this is due to some compiler activity.

Chandan Kumar
  • 4,570
  • 4
  • 42
  • 62
GreyCloud
  • 3,030
  • 5
  • 32
  • 47
  • 1
    C# is not in the picture at all, the debugger has its own expression evaluator. Exactly how it works is not documented at all. Some day it might start using Roslyn, it is unclear when. It doesn't necessarily have to cast to *double* to get that result, this might easily be a side-effect of the extended 80-bit internal precision that the FPU uses. [Intel's billion dollar mistake](http://stackoverflow.com/a/14865279/17034). You'll have to call it "unspecified behavior". – Hans Passant Oct 20 '16 at 09:34
  • thanks that's very interesting - i'd always assumed it ran C# in an interactive mode – GreyCloud Oct 20 '16 at 13:28

0 Answers0