0

I program in VB6 more than 15 years and today I got a very mysterious bug that made me investigate my variables and I was shocked also when I did this in the immediate window too.

VB6 Immediate Window:

?18.21-18.29
-7,99999999999983E-02 
?18.25-18.20
5,00000000000007E-02

In the immediate window of vb.net i do not have this problem:

?18.21-18.29
-0.0799999999999983
?18.25-18.20
0.050000000000000711

So if if numbers like these are assigned to variables in VB6 (tried Single, Double, even Variant - had the same outcome) I get disasterous results.

Am I going crazy over here?? Am I missing something?

C. Valas
  • 17
  • 2
  • 1
    What exactly is the problem? Scientific notation? Localization? What is wrong? – John3136 Jul 01 '16 at 00:32
  • No, I am just comparing two audio files that have 18.21 and 18.29 secs duration and want to get the difference. So expecting a -0.08 or -0.079 i get a -7.9 result!! I could multiply the numbers by 100 i suppose, make the calculation and then divide the outcome to get the proper result as a workaround, but is this valid? Is it explained anywhere why this is the result of the subtraction of two very close singles? – C. Valas Jul 01 '16 at 00:40
  • 3
    `E-02` is what you're missing. -7.99999999999983E-02 = -0.0799999999999983 – Jim Hewitt Jul 01 '16 at 00:53
  • 3
    This isn't a duplicate. Not what the OP was asking. – Jim Hewitt Jul 01 '16 at 01:15
  • 2
    This is not a duplicate. The user is actually having floating point to string formatting / conversion issues. To the OP, be aware that the immediate window is effectively converting that numeric output to a string. What you need to do is explicitly format the output to see better results. – tcarvin Jul 01 '16 at 12:09

0 Answers0