2

I have big integers in database (numeric(24,0)). And I want to convert this number to "human readable" format. I have function:

def from_int(value, precision):
    fprec = "%." + str(precision) + "f"
    return fprec % (Decimal(value) * (Decimal(10)**(-1*precision)))

And it works:

from_int(1000000, 6)
'1.000000'
from_int(1000000, 8)
'0.01000000'
from_int(1000000, 12)
'0.000001000000'
from_int(1000000, 2)
'10000.00'

but for:

from_int(19999999999999999, 2)
'200000000000000.00'

How to set precision without rounding?

Nips
  • 13,162
  • 23
  • 65
  • 103
  • I think you are using `precision` wrong. It’s more a `numberOfDecimalPlaces` or something. – poke Jun 02 '14 at 12:34

3 Answers3

2

Formatting with %f converts the number to floating point losing precision. Instead use str:

def from_int(value, precision):
    return str(Decimal(value) * Decimal(10)**(-precision))
Janne Karila
  • 24,266
  • 6
  • 53
  • 94
1

The problem is in the %f construct that expects a float. So even if you compute everything in decimal, you have a nasty float conversion at the end => so the rounding issue. I think you could just write :

def from_int(value, precision):
    return str((Decimal(value) * (Decimal(10)**(-1*precision))))

I tested it on your values and it gives correct results.

Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
0

If all you want is a string representation, you may be able to get away with something like this:

>>> ('%%.%df' % 2) % 2.445
'2.44'

so in your original code:

def from_int(value, precision):
    return ('%%.%df' % precision) % value

EDIT:

To deal with precision issue, it's indeed simpler to use Decimal type, and the quantize method:

>>> d = decimal.Decimal(19999999999999999)
>>> prec = 2
>>> str(d.quantize(decimal.Decimal(10) ** -prec))
'19999999999999999.00'
>>>

ref: How can I format a decimal to always show 2 decimal places?

EDIT: Using the Decimal type, you can shift decimal places without rounding:

>>> str(d * (decimal.Decimal(10) ** -prec))
'199999999999999.99'
Community
  • 1
  • 1