3

I'm guessing this is due to floating point and accuracy but I just wanted to check to ensure that there's nothing else that I'm missing.

I've got a float (stored in info.Amount) of 1007.62 this gets multiplied by 100 and cast to a long. At this point I get a value of 100761.

enter image description here

Is this just the good old rounding error that we encounter with Doubles being applied? The only way I can think this could happen is if underneath it's actually stored as 1007.6199999999999 (but rounded for display purposes). Then after the multiplication the conversion to a long ignores everything after the decimal point?

Ian
  • 33,605
  • 26
  • 118
  • 198
  • 4
    seems this was discussed/answered before, see here: http://stackoverflow.com/questions/8911440/strange-behavior-when-casting-a-float-to-int-in-c-sharp – 03Usr Apr 08 '14 at 11:32
  • @03Usr: Thanks - Didn't come up with anything when I searched/similar questions. – Ian Apr 08 '14 at 11:35

2 Answers2

4

You can use Convert.ToInt64 instead. This produces the correct value.

LinqPad-Testcode:

float z = 1007.62f;
z.Dump();

float x = z *100;
x.Dump();

long l = (long) (z*100);
l.Dump();

l = Convert.ToInt64(z*100);
l.Dump();

l = Convert.ToInt64(x);
l.Dump();

Result is:

1007,62
100762
100761
100762
100762

The cast (long) (z*100) uses the CIL-instruction conv.i8 that converts to Int64, pushes Int64 on stack, while Convert.ToInt64 uses Math.Round(double) to convert/cast to Int64.

A special note on conv.i8 is

Conversion from floating-point numbers to integer values truncates the number toward zero.

Jehof
  • 34,674
  • 10
  • 123
  • 155
  • Indeed - that's actually the fix that I opted for as you can see in my screenshot I checked a couple of different approaches. – Ian Apr 08 '14 at 11:42
  • @Ian, ah ok i didn´t noticed that in your image. – Jehof Apr 08 '14 at 11:43
1

Actually I think this is due to the variable you might declared.

Take the following sample:

       double amount = 1007.62;
       long result = (long) (amount*100);

result absolutely will be 100762 without any fraction or roundness. As a matter of fact, 1007.62 is a double number which should not be stored as float that may cause further problems like your situation.

Ali Dehghan
  • 462
  • 3
  • 6