8

I'm working with nested JSON-like data structures in python 2.7 that I exchange with some foreign perl code. I just want to 'work with' these nested structures of lists and dictionaries in amore pythonic way.

So if I have a structure like this...

a = {
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 66 }],
}

...I want to be able to deal with it in a python script as if it was nested python classes/Structs, like this:

>>> aa = j2p(a)  # <<- this is what I'm after.
>>> print aa.x
4
>>> aa.z = 99
>>> print a
{
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 66 }],
    'z': 99
}

>>> aa.y[2].b = 999

>>> print a
{
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 999 }],
    'z': 99
}

Thus aa is a proxy into the original structure. This is what I came up with so far, inspired by the excellent What is a metaclass in Python? question.

def j2p(x):
    """j2p creates a pythonic interface to nested arrays and
    dictionaries, as returned by json readers.

    >>> a = { 'x':[5,8], 'y':5}
    >>> aa = j2p(a)
    >>> aa.y=7
    >>> print a
    {'x': [5, 8], 'y':7}
    >>> aa.x[1]=99
    >>> print a
    {'x': [5, 99], 'y':7}

    >>> aa.x[0] = {'g':5, 'h':9}
    >>> print a
    {'x': [ {'g':5, 'h':9} , 99], 'y':7}
    >>> print aa.x[0].g
    5
    """
    if isinstance(x, list):
        return _list_proxy(x)
    elif isinstance(x, dict):
        return _dict_proxy(x)
    else:
        return x

class _list_proxy(object):
    def __init__(self, proxied_list):
        object.__setattr__(self, 'data', proxied_list)
    def __getitem__(self, a):
        return j2p(object.__getattribute__(self, 'data').__getitem__(a))
    def __setitem__(self, a, v):
        return object.__getattribute__(self, 'data').__setitem__(a, v)


class _dict_proxy(_list_proxy):
    def __init__(self, proxied_dict):
        _list_proxy.__init__(self, proxied_dict)
    def __getattribute__(self, a):
        return j2p(object.__getattribute__(self, 'data').__getitem__(a))
    def __setattr__(self, a, v):
        return object.__getattribute__(self, 'data').__setitem__(a, v)


def p2j(x):
    """p2j gives back the underlying json-ic json-ic nested
    dictionary/list structure of an object or attribute created with
    j2p.
    """
    if isinstance(x, (_list_proxy, _dict_proxy)):
        return object.__getattribute__(x, 'data')
    else:
        return x

Now I wonder whether there is an elegant way of mapping a whole set of the __*__ special functions, like __iter__, __delitem__? so I don't need to unwrap things using p2j() just to iterate or do other pythonic stuff.

# today:
for i in p2j(aa.y):
     print i
# would like to...
for i in aa.y:
     print i
martineau
  • 119,623
  • 25
  • 170
  • 301
Susanne Oberhauser
  • 413
  • 6
  • 13
  • I think you are looking for this solution - http://stackoverflow.com/questions/4984647/accessing-dict-keys-like-an-attribute-in-python#answer-14620633 – Yuri Astrakhan Aug 14 '14 at 22:27

3 Answers3

15

I think you're making this more complex than it needs to be. If I understand you correctly, all you should need to do is this:

import json

class Struct(dict):
    def __getattr__(self, name):
        return self[name]

    def __setattr__(self, name, value):
        self[name] = value

    def __delattr__(self, name):
        del self[name]

j = '{"y": [2, 3, {"a": 55, "b": 66}], "x": 4}'

aa = json.loads(j, object_hook=Struct)

for i in aa.y:
    print(i)

When you load JSON, the object_hook parameter lets you specify a callable object to process objects that it loads. I've just used it to turn the dict into an object that allows attribute access to its keys. Docs

Yuri Astrakhan
  • 8,808
  • 6
  • 63
  • 97
Thomas K
  • 39,200
  • 7
  • 84
  • 86
  • This is an interesting approach. However it looks like I loose the underlying nested `dict()` of `list()` of `dict()` structure, it's never constructed even. And I depend on that. – Susanne Oberhauser Apr 05 '12 at 12:28
  • @SusanneOberhauser: I'm not quite sure what you mean. It will simply be `Struct()` of `list()` of `Struct()` instead. `isinstance(aa, dict)` should still work, as Struct subclasses dict, and you can still use `aa['y']` notation if you need it. It seems like it would be easy to adapt code to that. – Thomas K Apr 09 '12 at 21:55
  • I realized that if I add nested substructures as a `Struct` instead of a `dict`, I get close to what I intend, as long as there is no name clashes between `dict` attributes and the dictionary attributes. `aa.items` is a built-in method for `Struct`, but it's a key in the dict for `_dict_proxy`. so `aa.copy = 44` works as intended in the latter, but not in the former. I think I'd really like to understand how to map a whole set of member functions to the proxied object using python meta programming. – Susanne Oberhauser Apr 10 '12 at 12:49
  • Yes, I've deliberately used `__getattr__` rather than `__getattribute__`, because overriding default methods opens up an ugly can of worms. If this is just for JSON, I'd either put up with having to do `aa['items']`, or specifically override a few common names to behave as keys rather than methods. If you're more interested in the programming theory, you might want to open up a new question just about that. – Thomas K Apr 10 '12 at 17:23
5

There is an attrdict library that does exactly that in a very safe manner, but if you want, a quick and dirty (possibly leaking memory) approach was given in this answer:

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

j = '{"y": [2, 3, {"a": 55, "b": 66}], "x": 4}'
aa = json.loads(j, object_hook=AttrDict)
Community
  • 1
  • 1
Yuri Astrakhan
  • 8,808
  • 6
  • 63
  • 97
1

I found the answer: There is intentionally no way to automatically map the special methods in python, using __getattribute__. So to achieve what I want, I need to explicitely define all special methods like __len__ one after the other.

Susanne Oberhauser
  • 413
  • 6
  • 13