How to create a JSON encodable dynamic string in Python

Question

I want to create an object that 'acts' like a string but when it's accessed, fires a function and returns that result.

The simple case for __str__ and __repr__ is easy enough; but I can't work out how to get json.dumps to treat it the same way;

import json, datetime


class DynamicString(str):
    def __init__(self, genf):
        self.generate = genf
    def __repr__(self):
        print("Called Repr")
        return self.generate()
    def __str__(self):
        print("Called Str")
        return self.generate()


dater=DynamicString(lambda: datetime.datetime.now().isoformat())
print(f"{dater!s}, {dater!r}, {dater!a}")
>>> Called Str
>>> Called Repr
>>> Called Repr
>>> 2019-05-01T13:52:12.588907, 2019-05-01T13:52:12.588933, 2019-05-01T13:52:12.588950

print(json.dumps(dater))
>>> "<function <lambda> at 0x10bb48730>"

It appears that however json.dumps is evaluating the object it's ignoring the custom dunder methods.

I can't use a custom JSONEncoder as this object is intended to be sent through modules I don't have access to modify. Any ideas?

UPDATE FOR CLARITY:

Expected output

json.dumps(dater)
>>> '"2019-05-01T16:43:21.956985"'

i.e. "Exactly as if it was just a normal string, but based on the current time"

These posts can help: https://stackoverflow.com/a/38764817/9609843 https://stackoverflow.com/q/18478287/9609843 — sanyassh, May 01 '19 at 13:13
Is there a particular reason you inherit from `str` that is not shown here? Or can you inherit from something else? — Error - Syntactical Remorse, May 01 '19 at 13:45
@Error-SyntacticalRemorse: It's not shown because `str` is a Python built-in. — martineau, May 01 '19 at 13:48
@martineau I understand that. But if he is not using anything from the parent of `str` then he does not need to inherit from it and instead can inherit from `JSONEncoder`. — Error - Syntactical Remorse, May 01 '19 at 13:49
Bolster: What do you _expect_ the resulting JSON output to be for this kind of object? Regardless, if you look at my answer to [Making object JSON serializable with regular encoder](https://stackoverflow.com/questions/18478287/making-object-json-serializable-with-regular-encoder), it shows how to monkey-patch the `json` module which will affect its usage in other modules. — martineau, May 01 '19 at 13:50
@Error-SyntacticalRemorse: I see. Yes, deriving the custom class from `str`, which is something the `json` module thinks it already knows how to handle, does make intercepting them difficult, if not impossible. — martineau, May 01 '19 at 13:59
I couldn't find a way to use just `dumps` but if you can change the `default` within `dumps` then this solution works: [Encoding nested python object in JSON](https://stackoverflow.com/a/5160278/8150685). But at that point you might as well just use `str` or `repr`. — Error - Syntactical Remorse, May 01 '19 at 14:10
Can't you do `json.dumps(str(dater))`? Or build your own serialization method in case more complex behavior is expected? — Rocky Li, May 01 '19 at 14:12
Bolster: Also, do you expect to get back a `DynamicString` instance when the JSON produced is later read back via `json.loads()`? — martineau, May 01 '19 at 14:26
Updated to explain expectations more clearly. Basically I want a "magic" string that looks like a string, walks like a string, and talks like a string, except it's really a "clock" (in this example). Nosying through the `json.JSONEncoder` core code it looks like this just may not be possible ¯\_(ツ)_/¯ — Bolster, May 01 '19 at 15:47
@Error-SyntacticalRemorse: Changing the `default` within `dumps` won't work because `default` is only used for objects the `JSONEncoder` doesn't already know how to handle (see [table](https://docs.python.org/3/library/json.html#json.JSONEncoder)). In other words it won't be used because `isinstance(dater, str)` is `True`. It works in the answer you linked to because class `Doc` isn't derived from something shown in the table. — martineau, May 02 '19 at 12:04
Bolster: From your last comment, it sounds like your class doesn't need to be derived from `str`, just act more-or-less like one. If that's true, there may be hope. However, you still haven't said what you would like or expect to get back from using `loads()` on the JSON output produced. — martineau, May 02 '19 at 12:09
@martineau would be happy for the `loads()` output to be treated like a normal static string; just want it to be enumerated in the repl as a 'dynamic string' like object. — Bolster, May 07 '19 at 15:50
Bolster: Can you be more specific about what you mean by "enumerated in the repl as a 'dynamic string' like object"? — martineau, May 07 '19 at 16:01

score 0 · Answer 1 · answered May 01 '19 at 18:10

The problem with this seem to lie with the fact that str class is immutable, which gets its value from initiation of your object, and does not change. __new__ function of class str is called, which takes in your genf, which then evaluated to

"<function <lambda> at 0xADDRESS>"

All subsequent call on the instance has no effect on this string. To verify this, you can overwrite the string at the beginning of your class definition by overwriting __new__:

class DynamicString(str):
    def __new__(cls, genf):
        o = super(DynamicString, cls).__new__(cls, "HAHA")
        return o
    ... rest of your class ...

... rest of your script ...

# Then:
>>> json.dumps(dater)
>>> "HAHA"

There doesn't seem to be any workaround as str is simply immutable, and by extension, your objects parent string is also immutable for the life of your instance..

I don't think the problem is immutability, it's because `JSONEncoder` already know how to handle objects of type `str` including any classes derived from it (i.e. `str` subclasses). There's a [table](https://docs.python.org/3/library/json.html#json.JSONEncoder) in the document showing the object types it handles automatically. — martineau, May 02 '19 at 12:23
The point being that when it does that it automatically takes the immutable string that is initiated on the creation of the instance. There's no way to change that, so the only choice left is to use default function like `str(instance)`. — Rocky Li, May 02 '19 at 13:32

score 0 · Answer 2 · answered May 07 '19 at 20:31

If all you want to do is display it like a 'dynamic string' object when json.dumps() is called, the first part of my answer to the question Making object JSON serializable with regular encoder could be used provided you don't derive your class from the built-in str class and add a to_json method.

Changing the former prevent the json module from automatically handling it because strings—and any subclass of it—are among the types of the things it's hardcoded to handle).

# Monkey-patch json module.
from json import JSONEncoder

def _default(self, obj):
    return getattr(obj.__class__, "to_json", _default.default)(obj)

_default.default = JSONEncoder.default  # Save unmodified default.
JSONEncoder.default = _default # Replace it.


if __name__ == '__main__':

    # Sample usage.

    import datetime
    import json


    class DynamicString(object):
        def __init__(self, genf):
            self.generate = genf

        def __repr__(self):
            print("Called Repr")
            return self.generate()

        def __str__(self):
            print("Called Str")
            return self.generate()

        def to_json(self):  # Added.
            return repr(self)


    dater = DynamicString(lambda: datetime.datetime.now().isoformat())
    print(f"{dater!s}, {dater!r}, {dater!a}")
    print()
    print(json.dumps(dater))

Output:

Called Str
Called Repr
Called Repr
2019-05-07T13:11:32.061129, 2019-05-07T13:11:32.061129, 2019-05-07T13:11:32.061129

Called Repr
"2019-05-07T13:11:32.061129"

How to create a JSON encodable dynamic string in Python

2 Answers2