-1

Edit: this is in python 2.7.7

Has anyone come across this? I need to serialise an key-value pair object into a string, in which each key contains a float.

There can be any number of entries (I demonstrate just one, "Test", in this example).

It seems that when evaluating the whole object, Python can add decimals to the value, whereas when we look at those exact same values independantly (outside of the object), they appear "normal".

I need a way to make the final, serialised object contain the original values (0.08 in this case) ; or a means to force them to 2DPs.

userConstraints = {
        "test": 0.08
    }

print userConstraints["test"] # Appears ok here
print str(userConstraints) # Has extra DPs here. Why? How to fix?

Output:

0.8
{'test': 0.080000000000000002}

Serialisation logic (on the other side)

obj=ast.literal_eval(theSerialisedString)

Output of serialised obj

{'test': 0.080000000000000002}
Janine Rawnsley
  • 1,240
  • 2
  • 10
  • 20
  • [That code doesn't produce that output.](https://ideone.com/DlruUT) – user2357112 Feb 03 '22 at 10:32
  • Different code, where `0.08` is a computed value, might produce that output. Python 2's `float.__str__` truncates values more aggressively that `float.__repr__`, and printing a container uses the `repr` of the container's contents. – user2357112 Feb 03 '22 at 10:34
  • @MarkRotteveel: Bad dupe. While floating point math is approximate, that fact alone is not enough to tell what's going on. It's not even apparent where any math might be happening here unless you understand the specifics of the situation in more detail. – user2357112 Feb 03 '22 at 10:35
  • @user2357112supportsMonica Can you try it in 2.7.7 ? – Janine Rawnsley Feb 03 '22 at 10:37
  • @user2357112supportsMonica The value `0.08` in double precision storage is approximately `0.08000000000000000166533453693773481063544750213623046875`, so the duplicate does apply, as the first answer explains this. – Mark Rotteveel Feb 03 '22 at 10:38
  • @MarkRotteveel Not really ; I'm aware of floating point math issues, but all I'm trying to do here is build a key-value pair object of keys with float arrays. I need to know how to force that last result to 2 DPs otherwise its sending garbage across – Janine Rawnsley Feb 03 '22 at 10:39
  • @MarkRotteveel it doens't answer why it show 0.08 in one case and not the other, and it doesn't tell me how to make the final result 2 DPs which is the requirement here – Janine Rawnsley Feb 03 '22 at 10:39
  • "send garbage across" – you aren't showing any (de)serialization here, so it's hard to tell what should be made to truncate floats to two decimal points. – AKX Feb 03 '22 at 10:43
  • @deceze Please look at the close log & comments – "is floating point broken" was a dupe for this until it was realized that it's a bad dupe and not directly relevant here. – AKX Feb 03 '22 at 10:45
  • @AKX Then add another one. You can’t expect any particular floating point number to have any specific exact value. End of story. – deceze Feb 03 '22 at 10:47
  • @deceze This post is about serialization (some serialization that we don't yet know of) that should be coerced to only do two decimals. – AKX Feb 03 '22 at 10:47
  • @AKX So “needs details” then…‽ – deceze Feb 03 '22 at 10:48
  • @JanineRawnsley So, IOW, please show your (de)serialization code, because that's what you're concerned with, not how Python 2.7 decides to print-represent floating-point numbers in their precise impreciseness. – AKX Feb 03 '22 at 10:54
  • @AKX I'm using print here for demonstration purposes only. That exact same output is being written to a string (a doc prop in spotfire), then sent across to a data function, and once deserialised back to an object it (naturally) contains the very long float with all the decimals – Janine Rawnsley Feb 03 '22 at 10:59
  • Please show the serialization code that forms the final data that is sent over the wire, not a "print for demonstration purposes". That's the only way we can help you with coming up with a serialization function that acts like you want. – AKX Feb 03 '22 at 11:01
  • 1
    "once deserialised back to an object it (naturally) contains the very long float with all the decimals" - however you resolve this, the deserialized output will probably be exactly the same output you're getting right now. Even if that output doesn't look like what you expected, it's probably correct. – user2357112 Feb 03 '22 at 11:02
  • @user2357112supportsMonica I was asked to round these values... perhaps an impossible request – Janine Rawnsley Feb 03 '22 at 11:08
  • @JanineRawnsley I noticed you had added "ironpython". Can you also please add the precise version of IronPython you're running on (if that's the case), so I don't need to spend more time trying to install CPython 2.7.7...? :) – AKX Feb 03 '22 at 11:12
  • 1
    The IronPython thing is an important detail that's probably the source of the `repr` discrepancies - IronPython has a completely different implementation of most of the language core. (The standard implementation should show the same `float.__repr__` output in all 2.7 releases. The Python 3.1 `float.__repr__` behavior was backported in 2.7.0.) – user2357112 Feb 03 '22 at 11:17
  • 3
    Now that we have more information, we can say for sure that the deserialized data structure is fine. These floats have exactly the values they should have. You're just seeing more of their digits than you're used to. If you want to see less digits, you can change your display handling, but the stored values are fine. – user2357112 Feb 03 '22 at 11:21

1 Answers1

2

So, all in all, your version and implementation of Python has a very precise repr() for floating-point numbers. (We can tell it's not a vanilla CPython 2.7.7, since the shorter floating-point repr() from Python 3.1 has been backported into CPython 2.7.0, and that version doesn't do reprs like you say it does.)

Your wire format for serialization is Python repr(), and you want to round off some of those lengthy floats, because reasons.

That's no problem, since Python comes with reprlib - a module for customizing repr (confusingly repr in Python 2).

Et voilà:

import repr as reprlib  # Python 2
import reprlib          # Python 3


class TruncatedFloatRepr(reprlib.Repr):
    def repr_float(self, x, level):
        v = repr(x)
        if "." in v:
            decimals_after_dot = len(v.partition(".")[-1])
            if decimals_after_dot > 5:  # Too many decimals? Truncate!
                return "%.2f" % x
        # Otherwise, use the vanilla `repr`.
        return v


userConstraints = {
    "test": 0.08000000004,
    "hello": ["world", "beep", 3.0, "boop"],
    "one": ("two", 1.1),
}

print(TruncatedFloatRepr().repr(userConstraints))

The output is

{'hello': ['world', 'beep', 3.0, 'boop'], 'one': ('two', 1.1), 'test': 0.08}

and of course you might want to consider changing that 5 to something else, or maybe count the number of zeroes in the decimal part, or whatever suits your business logic fancy. As it is, this implementation will just always truncate numbers with too much decimal precision, and that's likely not what you really want.

AKX
  • 152,115
  • 15
  • 115
  • 172
  • Seems the version of ironpython I'm stuck with doesn't have this module. I think at this point I should just push back on the rounding request since the solution works fine as is anyway. – Janine Rawnsley Feb 03 '22 at 11:55
  • 1
    Well, `reprlib` itself is a pure-Python module, so if you need it, you can just vendor it from e.g. https://github.com/python/cpython/blob/2.7/Lib/repr.py – AKX Feb 03 '22 at 11:58
  • 2
    It's a naming issue. `reprlib` was called `repr` in Python 2, but that was a terrible name that clashed with the built-in `repr` function, so they renamed it in Python 3. – user2357112 Feb 03 '22 at 12:10
  • 1
    Just `import repr as reprlib`. – user2357112 Feb 03 '22 at 12:12
  • 1
    @user2357112supportsMonica Ah, yeah, you're correct. https://docs.python.org/2/library/repr.html Edited into the answer. – AKX Feb 03 '22 at 12:30