0

Today I found out that casting int to float have pretty interesting precision and rounding rules. I have couple of numbers which gives me unexpected results:

float(1641016854000000000) -> 1.641016854e+18 (as expected)
float(1641016854000000100) -> 1.641016854e+18 (expected 1.6410168540000001e18)
float(1641016854000000200) -> 1.6410168540000003e+18 (expected 1.6410168540000002e+18)
float(1641016854000000300) -> 1.6410168540000003e+18 (as expected)
float(1641016854000000400) -> 1.6410168540000005e+18 (expected 1.6410168540000004e+18)

So my questions are:

  1. How I can prevent rounding of the values?
  2. How to get expected results instead those I have now?
alexfilexpo
  • 15
  • 2
  • `float`s have only so much precision; if you need more, use another type. – Scott Hunter Jun 30 '23 at 15:40
  • Does this answer your question? [Increasing floating point precision in Python](https://stackoverflow.com/questions/21079282/increasing-floating-point-precision-in-python) – TheDeafOne Jun 30 '23 at 15:45
  • stack**overflow** is full of information around the limitations (if they considered as such) of *float*. If you need higher precision consider: https://docs.python.org/3/library/decimal.html – DarkKnight Jun 30 '23 at 15:45
  • Here is another good writeup from python: [Floating Point Arithmetic: Issues and Limitations](https://docs.python.org/3/tutorial/floatingpoint.html). – Elias Jun 30 '23 at 15:56
  • 1
    You simply cannot represent `1.6410168540000003e-18` within `float` precision. Remember that both exponent and base are binary numbers? Try functions from [this answer](https://stackoverflow.com/questions/16444726/binary-representation-of-float-in-python-bits-not-hex): `float2bin` of your numbers, vary the value (add/subtract `0b1` manually) and `bin2float` back. This should explain why only `...2` and `...4` are generated, but not `...3`. – STerliakov Jun 30 '23 at 15:59
  • duplicate: [Why does the Python 3 string function appear to reduce precision of float?](https://stackoverflow.com/q/53750099/995714) – phuclv Jun 30 '23 at 16:00
  • @alexfilexpo `float` can represent about 2^64 different values exactly. `1641016854000000100` is not one of them. – chux - Reinstate Monica Jun 30 '23 at 16:04

1 Answers1

1

Python's default float type, as with most languages, can only be so precise to conserve memory. This is why you get floating point errors when working with extremely precise numbers. A good way to avoid this is to use python's decimal library.

TheDeafOne
  • 93
  • 10