How to avoid rounding float numbers in python?

Question

I have some json objects and when I convert them into dictionary in python they will rounded:

-112.07393329999999  ->  -112.0739333

My code is:

for line in open("c:\\myfile","r+").readlines():          
     d = json.loads(line)
     logtv = d['logtv']

That's running into floating point issues. If you need to keep that level of precision, you can't use Python's built in float. — David Robinson, Apr 22 '13 at 20:39
Duplicate of : http://stackoverflow.com/questions/7878341/python-dictionary-floats — Martin V., Apr 22 '13 at 20:40
You probably need to read [What Every Computer Scientist Should Know About Floating-Point Arithmetic](http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html) — Fred Larson, Apr 22 '13 at 20:40
possible duplicate of [Floating point inaccuracy examples](http://stackoverflow.com/questions/2100490/floating-point-inaccuracy-examples) — Aprillion, Apr 22 '13 at 20:43
json's double `{"key": -112.07393329999999}` is converted to exactly the same bits as `{"key": -112.0739333}`. use `{"key": "-112.07393329999999"}` in the json if you want to preserve the string as it is and not convert the string into double — Aprillion, Apr 22 '13 at 20:47
This is actually more about the python 2.7/3.1 feature "The repr() of a float x is shorter in many cases: it’s now based on the shortest decimal string that’s guaranteed to round back to x". — Pavel Anossov, Apr 22 '13 at 20:50

Pavel Anossov · Answer 1 · 2013-04-22T21:08:26.630

6

It's just a more compact representation of the same (binary, 64-bit) number.

In [12]: -112.07393329999999 == -112.0739333
Out[12]: True

In [17]: struct.pack('d', -112.0739333)
Out[17]: 've\xbcR\xbb\x04\\\xc0'

In [18]: struct.pack('d', -112.07393329999999)
Out[18]: 've\xbcR\xbb\x04\\\xc0'

If you want this exact decimal, there's no way to represent it as a number in JSON. You'll have to use strings in your JSON, Decimals/strings in your python and decimal fields in your DB.

SQL Server Floats are at most 64-bit (53-bit significand) as well.

edited Apr 22 '13 at 21:08

answered Apr 22 '13 at 20:39

Pavel Anossov

60,842
14
151
124

I need the exact value to be saved on my db – Mahdi Apr 22 '13 at 20:40
They are not the same number. – Esailija Apr 22 '13 at 20:41
@Mahdi: is your db column a double? They are exactly equal as binary doubles. – Pavel Anossov Apr 22 '13 at 20:45
1

@Mahdi you asked about json objects vs. python dictionaries, where does db come into play here? – Aprillion Apr 22 '13 at 20:45
@Esailija They are the exact same number - as the invocation of `struct.pack` in the answer shows, their memory representation is identical to the last bit. – user4815162342 Apr 22 '13 at 20:46
2

@Esailija: they are equal as IEEE-754 doubles, but not quadruples. – Pavel Anossov Apr 22 '13 at 20:47
@deathApril I read the json object and after text processing I will import them into database – Mahdi Apr 22 '13 at 20:49
3

@PavelAnossov you said that they are equal numbers.. but they are only equal IEEE-754 doubles which is not the same thing as a number. sigh. – Esailija Apr 22 '13 at 20:49
@PavelAnossov my db is sql server and column type is float (16 digit supported) – Mahdi Apr 22 '13 at 20:50
@Esailija: they are equal as both "python floats" and "JSON numbers". Saying "IEEE-754 doubles" instead of "numbers" would just confuse OP more. – Pavel Anossov Apr 22 '13 at 20:52
I just realized that "float(16)" in SQL Server actually means _16 bits_ (actually rounded up to 24 bits), not _16 decimal digits_. So, it's not even an IEEE double, but a single… – abarnert Apr 22 '13 at 21:07

cmd · Answer 2 · 2013-04-25T19:30:32.843

3

The answer is don't use floats. In most languages floats only have about 6 digits of significance and not too many more for doubles (note python floats are doubles). Use decimals if you know the exact precision. For JSON send it as a string or an int with an implied decimal point.

soap box: floats are very much overused. Floats should not be used by anything that you wouldn't represent with scientific notation, as that is what they really are underneath.

Note: Databases do not usually use floating point numbers, they use fixed point numbers. Which is exactly what a decimal is.

clarification

before you write the json file do something like

with  open("c:\\myfile","w") as my_file:
    for key in d:
        if isinstance(d[key], Decimal):
            d[key] = str(d[key])
    my_file.write(json.dumps(d))

then when reading the json, you can just put the value into the database as is or convert it back to Decimal if you need to work with it more.

edited Apr 25 '13 at 19:30

answered Apr 22 '13 at 20:51

cmd

5,754
16
30

1

just to clarify, `-112.07393329999999` doesn't look like a 17 digits precision number, so decimals should be used BEFORE sending the number through json, not only when processing it like it is now – Aprillion Apr 22 '13 at 20:56
@deathApril right, my point was don't turn the value to float at any point. Decimals are great at getting generated strait from strings. – cmd Apr 22 '13 at 21:37
Not surprisingly, in Ruby the same problems occur and the proposed solutions are going in the same direction as Python's. It is even advocated that [Float is Legacy](http://www.slideshare.net/mrkn/float-is-legacy). – steenslag Apr 22 '13 at 22:35

How to avoid rounding float numbers in python?

2 Answers2