0

I am trying to evaluate numbers upto 3 decimal precision. The formulas are fairly simple but sum of differences between values and difference between sum of values are not equal.

The dataset contains nearly 2050 entries.

result_column_1 = SUM(column1) - SUM(column2)
result_column_2 = SUM(column1-column2)

I am getting a huge dissimilarity between result_column_1 and result_column_2

column1 and column2 are already truncated (not rounded) to 3 decimal places.

1. ROUND() - didn't work
2. CAST TO Demical, Numeric - didn't work
3. TRUNC() - didn't work
Pompedup
  • 566
  • 2
  • 8
Adi
  • 13
  • 3
  • 1
    Please, add your table description and maybe some entries :) – Pompedup Nov 22 '22 at 01:50
  • Does this answer your question? [Is floating point math broken?](https://stackoverflow.com/questions/588004/is-floating-point-math-broken) – Ken White Nov 22 '22 at 01:58
  • For fewer entries, it did result in the same answer!! – Adi Nov 22 '22 at 01:58
  • @KenWhite, I believe that's exactly the issue but couldn't find appropriate answers. – Adi Nov 22 '22 at 02:03
  • That **is** the appropriate answer. Floating point math isn't always exact, and especially when summing many values, the minor differences add up, causing larger errors. – Ken White Nov 22 '22 at 02:08
  • That **is** the appropriate answer. Floating point math isn't always exact, and especially when summing many values, the minor differences add up, causing larger errors. – Ken White Nov 22 '22 at 02:08
  • @KenWhite, is there any way that I can avoid or reduce it? The values being used in further multiplication calculations result in a colossal mismatch. – Adi Nov 22 '22 at 02:26

0 Answers0