Is there an equivalent to @property for containers in Python?

Question

I'm writing a simplified wrapper class in Python for an AWS module (Boto, specifically). Several times in this process I've used @property to avoid special "getter" and "setter" methods in my library - I'm told that this is the more pythonic way to do it. When using the class, the programmer call the methods as if they were simple objects, like this:

myclass.myprop = 5         # sends "5" to myprop's setter function
result = myclass.myprop    # calls myprop's getter function and stores the result

But I'm also dealing with several sets of objects - name/value pairs of tags, for example - that I would like to access as if they were held in a container, possibly a dictionary or a list. Taking the tag example:

myclass.tags["newkey"] = "newvalue"   # runs a function that applies tag in AWS
result = myclass.tags["newkey"]       # accesses AWS to get value of "newkey" tag

From what I'm seeing, it looks like it would be possible to do this by subclassing dict, but I feel like I'm missing something here. What is the most pythonic way to create an interface like this?

EDIT: I ended up using Silas Ray's solution, but modified it so that the classes can be used to define multiple dict-like objects. It's not exactly clean, but I'm going to post my modified code and an explanation here to help anyone else having trouble grokking this.

class FakeDict(object):

    def __init__(self, obj, getter, setter, remover, lister):
        self.obj = obj
        self.getter = getter
        self.setter = setter
        self.lister = lister
        self.remover = remover

    def __getitem__(self, key):
        return self.getter(self.obj, key)

    def __setitem__(self, key, value):
        self.setter(self.obj, key, value)

    def __delitem__(self, key):
        self.remover(self.obj, key)

    def _set(self, new_dict):
        for key in self.lister(self.obj):
            if key not in new_dict:
                self.remover(self.obj, key)
        for key, value in new_dict.iteritems():
            self.setter(self.obj, key, value)

class ProxyDescriptor(object):

    def __init__(self, name, klass, getter, setter, remover, lister):
        self.name = name
        self.proxied_class = klass
        self.getter = getter
        self.setter = setter
        self.remover = remover
        self.lister = lister

    def __get__(self, obj, klass):
        if not hasattr(obj, self.name):
            setattr(obj, self.name, self.proxied_class(obj, self.getter, self.setter, self.remover, self.lister))
        return getattr(obj, self.name)

    def __set__(self, obj, value):
        self.__get__(obj, obj.__class__)._set(value)

class AWS(object):

    def get_tag(self, tag):
        print "Ran get tag"
        return "fgsfds"
        # Call to AWS to get tag

    def set_tag(self, tag, value):
        print "Ran set tag"
        # Call to AWS to set tag

    def remove_tag(self, tag):
        print "Ran remove tag"
        # Call to AWS to remove tag

    def tag_list(self):
        print "Ran list tags"
        # Call to AWS to retrieve all tags

    def get_foo(self, foo):
        print "Ran get foo"
        return "fgsfds"
        # Call to AWS to get tag

    def set_foo(self, foo, value):
        print "Ran set foo"
        # Call to AWS to set tag

    def remove_foo(self, tag):
        print "Ran remove foo"
        # Call to AWS to remove tag

    def foo_list(self):
        print "Ran list foo"
        # Call to AWS to retrieve all tags

    tags = ProxyDescriptor('_tags', FakeDict, get_tag, set_tag, remove_tag, tag_list)
    foos = ProxyDescriptor('_foos', FakeDict, get_foo, set_foo, remove_foo, foo_list)


test = AWS()

tagvalue = test.tags["tag1"]
print tagvalue
test.tags["tag1"] = "value1"
del test.tags["tag1"]

foovalue = test.foos["foo1"]
print foovalue
test.foos["foo1"] = "value1"
del test.foos["foo1"]

Now for the explanation.

tags and foos are both class-level instances of ProxyDescriptor, and are instantiated only once when the class is defined. They've been moved to the bottom so they can reference the function definitions above them, which are used to define the behavior for the various dictionary actions.

Most of the "magic" happens on the __get__ method for ProxyDescriptor. Any code with test.tags will run the __get__ method of the descriptor, which simply checks if test (passed in as obj) has an attribute named _tags yet. If it doesn't, it creates one - an instance of the class that was passed to it before. This is where FakeDict's constructor is called. it ends up being called and created exactly once for every instance of AWS where tags is referenced.

We've passed the set of four functions through the descriptor and through FakeDict's constructor - but using them inside FakeDict is a little tricky because the context has changed. If we use the functions directly inside an instance of the AWS class (as in test.get_tag), Python automatically fills the self argument with the owner test. But they're not being called from test - when we passed them to the descriptor, we passed the class-level functions, which have no self to reference. To get around this, we treat self as a traditional argument. obj in FakeDict actually represents our test object - so we can just pass it in as the first argument to the function.

Part of what makes this so confusing is that there's lots of weird circular references between AWS, ProxyDescriptor, and FakeDict. if you're having trouble understanding it, keep in mind that in both 'ProxyDescriptor' and 'FakeDict', obj is an instance of the AWS class that has been passed to them, even though the instance of FakeDict lives inside that same instance of the AWS class.

The Pythonic way is to use neither getter and setter methods nor `@property`. The semantics of attribute access and assignment to attributes are sufficient by themselves. Unless! Unless you need to verify input or calculate output. Note that you can start out not using properties and switch to using `@property` without changing client code. So you only need to add it when you need it. — Steven Rumbalski, Sep 18 '13 at 20:12

score 3 · Answer 1 · edited May 23 '17 at 10:25

Implement the __getitem__ hook to hook into object[..] index or item access:

>>> class DuplexContainer(object):
...     def __init__(self):
...         self._values = ['foo', 'bar', 'baz']
...     def __getitem__(self, item):
...         if item in self._values:
...             return self._values.index(item)
...         return self._values[item]
... 
>>> d = DuplexContainer()
>>> d[1]
'bar'
>>> d['baz']
2

To support item assignment, you can implement __setitem__(), and deletion is handled by __delitem__().

You can also opt to support slicing; when someone uses slice notation on your custom object, the __*item__() hooks are passed a slice object, you can then return values, set values or delete values as required based on the slice indices:

>>> class DuplexContainer(object):
...     def __init__(self):
...         self._values = ['foo', 'bar', 'baz']
...     def __getitem__(self, item):
...         if isinstance(item, slice):
...             return ['Slice-item {}'.format(self._values[i]) 
...                     for i in range(*item.indices(len(self._values)))]
...         if item in self._values:
...             return self._values.index(item)
...         return self._values[item]
... 
>>> d = DuplexContainer()
>>> d[:2]
['Slice-item foo', 'Slice-item bar']

Don't forget `__setitem__`, and maybe `__delitem__`. – user2357112 Sep 18 '13 at 19:54 — user2357112, Sep 18 '13 at 19:54

Silas Ray · Accepted Answer · 2013-09-18T20:41:44.613

@Martjin Pieters is right on track with the __getitem__ (and __setitem__), but since I'm guessing you'll probably want your objects with the container interface to act as proxies for an underlying interface (AWS), and thus you're container hooks will need access to state from the containing object, you should look at writing a custom descriptor. propertys are actually descriptors themselves.

class AWSTagsProxy(object):

    def __init__(self, aws_inst):

        self.aws_inst = aws_inst

    def __getitem__(self, key):

        return self.aws_inst.get_tag(key)

    def __setitem__(self, key, value):

        self.aws_inst.set_tag(key, value)

    def __delitem__(self, key):

        self.aws_inst.remove_tag(key)

    def _set(self, tag_dict):

        for tag in self.aws_inst.tag_list():
            if tag not in tag_dict:
                self.aws_inst.remove_tag(tag)
        for tag, value in tag_dict.iteritems():
            self.aws_inst.set_tag(tag, value)

class ProxyDescriptor(object):

    def __init__(self, name, klass):

        self.name = name
        self.proxied_class = klass

    def __get__(self, obj, klass):

        if not hasattr(obj, self.name):
            setattr(obj, self.name, self.proxied_class(obj))
        return getattr(obj, self.name)

    def __set__(self, obj, value):

        self.__get__(obj, obj.__class__)._set(value)

class AWS(object):

    tags = ProxyDescriptor('_tags', AWSTagsProxy)

    def get_tag(self, tag):

        # Call to AWS to get tag

    def set_tag(self, tag, value):

        # Call to AWS to set tag

    def remove_tag(self, tag):

        # Call to AWS to remove tag

    def tag_list(self):

        # Call to AWS to retrieve all tags

This is more analogous to property setter and getter methods in any case, since your __setitem__ and __getitem__ have access to the containing instance (obj in ProxyDescriptor instance scope, aws_inst in AWSTagsProxy instance scope) similarly to the way that a property method has access to self.

This is very close to what I want, and I was able to get it working in my code, but I have no clue how it actually works. Can you explain in a little more detail what's going on underneath? For example, where is the constructor for AWSTagsProxy being called? — ghicks-rmn, Sep 19 '13 at 21:57
Also, the tags object is being called on the class level and not on __init__ - does that have implications for multiple instances of the class? And is there any way to pass functions into AWSTagsProxy instead of hard-coding them as "set_tag", "remove_tag", etc.? I would love to make them generic and just instantiate them for every dict-like construct in my wrapper class. Sorry for so many questions - just REALLY would love to understand descriptors better. — ghicks-rmn, Sep 19 '13 at 22:06
From your question edit, it looks like you pretty much got it. The reason I organized things the way I did was to make `ProxyDescriptor` more generic. The way I wrote it, the class to which you add the descriptor and the class that defines the proxy interface become loosely coupled companions, and the `ProxyDescriptor` is just a middle man that acts as a lazy loading factory for the proxy. Once you put more implicit knowledge of the underlying proxied data structure in the `ProxyDescriptor`, you make it less flexible. That could be fine for you, but I wanted a generic answer. — Silas Ray, Sep 20 '13 at 13:19
You could also write a metaclass that is in a sense proxy interface-aware, and then use decorators to identify methods in the class that would be dynamically constructed in to a proxy object which would then be automatically wrapped in a `ProxyDescriptor` attached to the user-created class, but then you have to be able to define a metaclass on every object you want to use this pattern on, which you may or may not be able to do, and you'd have to worry about resolving metaclass conflicts as well. The way it is here, you can use this pattern on any class you want. — Silas Ray, Sep 20 '13 at 13:25

Is there an equivalent to @property for containers in Python?

2 Answers2