-2

I've been very confused because this calculation gives different results.

Method 1: 

select 6000000*(95/365)*(3.00/100) = 46849.3149600000. Refer above image in link

Method 2: 

select ((6000000*95*3.00)/36500) = 46849.315068. Refer above image in link

When calculate using calculator getting same results for above two methods

46849.315068

Can someone explain me this weird behaviour?

Note I'm using Mysql aurora AWS.

user207421
  • 305,947
  • 44
  • 307
  • 483
MadhanKumar BujjingaRao
  • 1
  • 1
  • 1
    Your first calculation has many more operations than your second. Floating point arithmetic is subject to rounding errors on a computer. Your results are accurate to six 10,000ths. I'd say that's pretty good. See [Is floating point math broken?](https://stackoverflow.com/q/588004/14853083) – Tangentially Perpendicular Jun 28 '23 at 00:23
  • NB The result of your second calculation as typed is not `46849.315068` but `16438358.775342465753424657534247`. – user207421 Jun 28 '23 at 00:50
  • [tag:sql-query-store] has nothing to do with MySQL. – user207421 Jun 28 '23 at 01:25

1 Answers1

0

The difference in results between Method 1 and Method 2 is due to the way floating-point arithmetic is performed in MySQL and other programming languages.

Method 1: select 6000000*(95/365)*(3.00/100)

In this method, the division operations (95/365) and (3.00/100) are performed first, resulting in decimal approximations. Then the multiplication operations are performed. The intermediate results are rounded to a certain number of decimal places, which can lead to slight differences in the final result.

Method 2: select ((6000000*95*3.00)/36500)

In this method, the multiplication operations (6000000*95*3.00) are performed first. Since multiplication is generally more accurate than division in floating-point arithmetic, the intermediate result retains more precision. Then the division operation (result)/36500 is performed, which can also introduce a small rounding error.

The differences in precision and rounding can cause the two methods to produce slightly different results, even though they represent the same mathematical calculation.

ps-
  • 234
  • 1
  • 6