What method is used when accessing a dict as a whole?

Question

I subclassed dict in order to have a volatile version: one where its values dissapear when some time passes after the last update⁽¹⁾:

import time
import collections

class Dict10min(dict):
    def __init__(self):
        super().__init__(self)
        self.expire = 0     # when to expire

    def __getitem__(self, item):
        if time.time() < self.expire:
            val = dict.__getitem__(self, item)
            return val
        else:
            # purging the dict
            self.clear()
            raise KeyError("key {item} does not exist anymore".format(item=item))

    def get(self, item, d=None):
        if time.time() < self.expire:
            try:
                val = dict.__getitem__(self, item)
            except KeyError:
                return d
            else:
                return val
        else:
            # purging the dict
            self.clear()
            return None

    def __setitem__(self, key, value):
        # set expire time at 10 minutes from now
        self.expire = time.time() + 5       # 5 seconds for tests, will be 600
        dict.__setitem__(self, key, value)

    # solution for update is from @johnrsharpe at http://stackoverflow.com/a/30242574/903011
    def update(self, other=None, **kwargs):
        if other is not None:
            for k, v in other.items() if isinstance(other, collections.Mapping) else other:
                self[k] = v
        for k, v in kwargs.items():
            self[k] = v

a = Dict10min()
a[1] = 2
time.sleep(6)
print(a.get(1))
a[3] = 4
print(a.get(1))

b = Dict10min()
b[1] = 2
time.sleep(6)
try:
    print(b[1])
except KeyError:
    print("entry does not exist anymore")
b[3] = 4
try:
    print(b[1])
except KeyError:
    print("entry does not exist anymore")

This correctly outputs

None
None
entry does not exist anymore
entry does not exist anymore

The part which is missing is the one which handles the overall Dict10min object: when accessing it as a whole none of __getitem__ nor .get() is used and what is returned is the actual content of the dict (and not the time-altered version).

Where should I hook in order to capture the moment where my object is accessed as a whole (to return {})?

⁽¹⁾the code was updated following @TomDalton comment

@TomDalton: isn't `__repr__` a string? I would like to return an empty dict. — WoJ, Jan 23 '16 at 14:46
Accessing the dict doesn't go *through* the object's API, so there is nothing you can hook. — chepner, Jan 23 '16 at 16:56
The line `print("the whole dict: {i}".format(i=a))` is not accessing the dict directly, it is calling the method on the object to get its string representation. — Tom Dalton, Jan 23 '16 at 20:51
@TomDalton: ah ok, I understand what you mean. My example code should have made it explicit that the real one does a `return a` which is consumed by a `x = the_function_which_returns_my_time_boud_dict()` — WoJ, Jan 23 '16 at 20:54
A question on how you want this to work: at the moment, if you set `a[1] = 2`, then wait 6 secs, then do `a[1]` you get `None` (not `KeyError`). If you then set `a[3] = 4` and then immediately do `a[1]` again, it's original value will reappear. It's not clear if this is what you actually want. Can you give some more info on exactly what you're trying to achieve? — Tom Dalton, Jan 23 '16 at 21:00
@TomDalton: you are correct. I updated the code so that this is consistant and behaves like a dict (throws a `KeyError` correctly) - the dict is emptied when its time runs out — WoJ, Jan 23 '16 at 21:28
Instead of the loop[ around `dict.__delitem__(self, k)`, you can jsut call self.clear(). — Tom Dalton, Jan 23 '16 at 21:28
I had a play around with this, using `__getattribute__` to intectept any call into the dict to do the expiry. It did work, but it's pretty ugly. I'll post it as an answer but I would be very hesitant to actually use it in real life! — Tom Dalton, Jan 23 '16 at 21:29
@TomDalton: re: `self.clear()` - thanks, I did not know this method — WoJ, Jan 23 '16 at 21:32

score 1 · Accepted Answer · answered Jan 23 '16 at 21:57

The following seems to work for me. It's super hacky! You'd need to add hooks to any other 'write' methods you might use.

import time


class ExpiringDict(dict):
    def __init__(self, expiry_secs, *args, **kwargs):
        self.expire = time.time() + expiry_secs
        self.expiry_secs = expiry_secs

        super(ExpiringDict, self).__init__(*args, **kwargs)

    def _reset_expire(self):
        self.expire = time.time() + self.expiry_secs

    def _check_expire(self):
        if time.time() > self.expire:
            self.clear()

    def __setitem__(self, key, value):
        self._reset_expire()
        return super(ExpiringDict, self).__setitem__(key, value)

    def update(self, *args, **kwargs):
        self._reset_expire()
        return super(ExpiringDict, self).update(*args, **kwargs)

    def __getitem__(self, *args, **kwargs):
        self._check_expire()
        return super(ExpiringDict, self).__getitem__(*args, **kwargs)

    def __getattribute__(self, name):
        attr = super(ExpiringDict, self).__getattribute__(name)
        if name not in {"_check_expire", "expire", "clear"}:
            self._check_expire()
        return attr

It passes the same tests as you've got above. Sadly, __getitem__ is needed because of this: https://docs.python.org/3.5/reference/datamodel.html#special-lookup

As such, there might be other methods you'll need to add like this. As an example, if you want to print the dict as you had earlier, you need to add a repr like this:

def __repr__(self, *args, **kwargs):
    self._check_expire()
    return dict.__repr__(self, *args, **kwargs)

Which makes sure the expiry is checked before printing.

What method is used when accessing a dict as a whole?

1 Answers1