1

c# code:

float floatA = 4;
float floatB = 17;
float floatC = floatA / floatB; //0.235294119

double doubleA = 4;
double doubleB = 17;
double doubleC = doubleA / doubleB; // 0.23529411764705882
float res = (float)doubleC; // 0.235294119

js code:

var a = 4;
var b = 17;
var c = a / b; // 0.23529411764705882
var res = (a / b).toPrecition(9); // 0.235294118

Why floatC & res are 0.235294119 instead of 0.235294118?

doubleC in c# is the same as c in js. Basically I need to round numbers in javascript the same as c# does it(double to float)

Joel Coehoorn
  • 399,467
  • 113
  • 570
  • 794
Denis Gursky
  • 138
  • 5
  • 3
    http://stackoverflow.com/questions/11085052/round-twice-error-in-nets-double-tostring-method – adeneo Apr 11 '14 at 17:41
  • 1
    It seems likely that 1176 should be 118 when rounded, not 119, so the issue is probably in C#, not in javascript, which seems to get it right. – adeneo Apr 11 '14 at 17:45
  • 2
    The discrepancy is one part in ten billion. What are you measuring that requires that level of precision? Neither C# nor Javascript promise you that float computations will always be performed identically from run to run; so long as a minimum precision per computation is maintained, both languages are free to vary the actual results as they see fit. Instead of asking "how can I ensure that the results are identical", which you cannot, the question you should be asking is "how can I ensure that my program works correctly when an error of one part in ten billion is introduced?" – Eric Lippert Apr 11 '14 at 18:49

1 Answers1

0

As a float, 4/17 = 0.23529411852359771728515625. Rounded to 9 digits, that's 0.235294119.

As a double, 4/17 = 0.235294117647058820264049927573068998754024505615234375. Rounded to 9 digits, that's 0.235294118.

This is rounding as expected.

Rick Regan
  • 3,407
  • 22
  • 28