0

Edit3: Actual answer.

Two wrong assumptions lead to the question popping up:

a) The implicit cast for floating point numbers is to double, not float. This explains the difference between the two operations.

b) Don't trust the debugger. As was pointed out by Servy (after a bit of poking with a stick) the Debugger uses ToString() which as he says apparently does implicit rounding. Floating point values that have no accurate representation in binary are displayed rounded. One may argue about this, i would argue that it's evil.

Edit2: Accepted dupe: Strange behavior when casting a float to int in C#

Thank you Nolonar. The rest of you should read the actual question before posting.

Edit: This isn't really a duplicate, in fact the answer to the alleged duplicate merely points out why i am confused... i guess i should make myself more clear...

The question is,

a) why does (61.1 * 10) yield different results from (61.1f * 10) [assumed compiler optimization plays a role here]?

b) why does the second line in the example below appear as "621.0" in debugger output, when printing it, and so on - when, as apparent after casting to int, it is in fact closer to 620.9~? [assumed VS debugger "watch" feature does formatting magic here]

Ok, so this doesn't happen to me very often any more, but i am fundamentally confused here about what appears to be a simple thing (And therein lies my problem, assuming floats and casts are a simple thing..)

I mean, I know how floats work. But this is so far from the precision limit and presumably a non-critical operation?

I could guess that compiler optimization has to do with the two cases in the title yielding different results, but the real case is at runtime:

float x = 62.1
x*10 // == 621.0
(int)(x*10) // == 620

Am i correct in assuming that 621.0 as shown is really rather something like 620.99999999999999999999999999999998 and thus gets truncate after the cast?

In this case, i suppose i should blame the VS "watch" feature..

Community
  • 1
  • 1
Ben Kunz
  • 136
  • 4
  • 5
    `I mean, I know how floats work.` If you know how floats work then you should know that they are, for all intents and purposes, approximations of the value, not the actual value, and as such you should virtually never use `==` to compare them, as they are usually just "close" and not "equal". – Servy Feb 21 '13 at 15:30
  • The == is just for demonstration purposes. – Ben Kunz Feb 21 '13 at 15:33
  • 1
    @BenKunz In that case you should write `(int)(x*10) // == 620` to showcase the demonstration and to avoid misunderstandings. – Nolonar Feb 21 '13 at 15:35
  • 1
    Actually, I do not agree with the current duplicate. This question is more a duplicate of that one: [Strange behavior when casting a float to int in C#](http://stackoverflow.com/questions/8911440/strange-behavior-when-casting-a-float-to-int-in-c-sharp) – Nolonar Feb 21 '13 at 15:42
  • I love how people clearly are not reading the text, but just point to the nearest question about float precision as a dupe, or even (Servy) are being very nearly arrogant and at best unconstructive. Thank you Nolonar. – Ben Kunz Feb 21 '13 at 15:49
  • 1
    @BenKunz The question you edited in (`"why does the second line in the example below appear as "621.0" in debugger output"`) is not a duplicate for the question this is closed as a duplicate of, the question you had before was. As for the edited quetion, that's simple enough; the debugger prints the result of `ToString`, and `ToString` often performs some rounding when printing the value (often out of necessity) and so doesn't always indicate the underlying inaccuracies of the value. That floats are inaccurate is why you see the behavior of the code you first showed. – Servy Feb 21 '13 at 16:03
  • the question you had before was => the question i had before was exactly the same, you should read the entire question before posting, especially if you are going to be a dick about it. It's right in the title! Read the Title! – Ben Kunz Feb 21 '13 at 16:06
  • because `61.1f` is a `float` and `61.1` is a `double` – phuclv Dec 31 '15 at 06:48

0 Answers0