-2

So I was trying to minimize floating point errors when doing arithmetic in python and I stumbled upon the Decimal module of python. It worked great in the first up until this operation.

from decimal import *
getcontext().prec = 100

test_x = Decimal(str(3.25)).quantize(Decimal('0.000001'), rounding=ROUND_HALF_UP)
test_y = Decimal(str(2196.646351)).quantize(Decimal('0.000001'), rounding=ROUND_HALF_UP)

print((test_y)*(test_x**Decimal('2')))

The above code outputs 23202.077082437500000000 instead of 23202.07708 where it is the output of our usual conventional arithmetic calculator. How can I output it like our calculator with rounding off to 6 decimal places? Also do you have better ways to do arithmetic calculations in python?

I have tried the round() function of the python but that is off limits for me because I am dealing with very large numbers which reaches the maximum length of numbers that the round() function support

Adding further context to the code. I cant change the value of getcontext().prec and the .quantize(Decimal('0.000001')) because I am dealing with numbers like 109796940503037.6545639765 and it is giving me errors if I dont set getcontext().prec to a high number.

I can't change the getcontext().prec to let's say 6 because it always gives the error: InvalidOperation: [<class 'decimal.InvalidOperation'>]

saladlord
  • 1
  • 1
  • 1
    FWIW, you're already losing precision at `str(2196.646351)`. You need to write `'2196.646351'`. – deceze Jan 26 '23 at 11:05
  • What would you expect `Decimal('0.1').quantize(Decimal('0.1'))` to be? It is just the same as `Decimal('0.1')`, right? `quantize` has no effect here, since it means `0.1 but with the exponent of 0.1`. Next question: what would you expect `Decimal('0.1')*Decimal('0.1')` to be? Surely not 0! That would make that whole Decimal library quite useless. You expect the result to be `Decimal('0.01')` I surmise. So, you should understand that exponent of the operands are the the one of the result. In other words, you can `quantize` `test_x` and `test_y` all you want, it doesn't bind your result. – chrslg Jan 26 '23 at 11:15
  • Sorry it may actually lack context but those .quantize(Decimal('0.000001') functions are important because this code is inside a function where i am dealing with numbers like 109796940503037.6545639765 so the quantize function automatically rounds off to 6 decimal places. I just simplified it here because it may not be necessary to mention – saladlord Jan 26 '23 at 11:30
  • @deceze Those two strings are equal. The precision lost from source code to `float` is "recovered" from `float` to string. – Kelly Bundy Jan 26 '23 at 12:42

1 Answers1

-1

If you do: Decimal(str(123312.12321221332)) it converts a float to string and the passes it to Decimal and you are losing precision during that conversion.

Do: Decimal('123312.12321221332') instead.

Also keep in mind that:

Unlike hardware based binary floating point, the decimal module has a user alterable precision (defaulting to 28 places) which can be as large as needed for a given problem

https://docs.python.org/3/library/decimal.html

Axeltherabbit
  • 680
  • 3
  • 20
  • Thanks for the tip although it still did not solve my problem. Will using f strings achieve the same as Decimal('123312.12321221332')? like this x = 123312.12321221332 Decimal(f'{x}') – saladlord Jan 26 '23 at 11:38
  • No. The problem is that the `x` value **already lost** the precision. If you want a `Decimal` value that represents a decimal value exactly, with a specific number of decimal digits, you must **start with** a string. Like deceze told you in the first comment on the question. – Karl Knechtel Jan 26 '23 at 11:41
  • @KarlKnechtel Not necessarily. For example, the two values in this answer are equal (so the answer doesn't make sense). – Kelly Bundy Jan 26 '23 at 12:29
  • @KellyBundy "lost precision" does not entail "different value". – Karl Knechtel Jan 26 '23 at 12:32
  • @KarlKnechtel But the "don't do this, do *this* instead" suggests a different value. – Kelly Bundy Jan 26 '23 at 12:36
  • @KellyBundy my point is that you should not assume that they will be equal, that's not true for every number – Axeltherabbit Jan 26 '23 at 13:58