Is it possible to apply my own decorators to builtin methods in Python?

Question

I've just come across Python decorators. Just out of interest, can you apply your own decorator to a built-in object method somehow? Say I wanted to apply this:

def remove_empty(fn):
    def filtered():
        return filter(lambda x: x != '', fn())
    return filtered

To this:

some_string.split('\n')

in order to remove empty strings. Is it possible? Or even a good idea?

You want the split() method to always remove empty strings on every call in your application? — Vaughn Cato, Sep 18 '12 at 11:50
Good point. It might be a bad example, but I'd still like to know if it's possible... — muskrat, Sep 18 '12 at 11:51

senderle · Accepted Answer · 2018-06-15T18:25:47.243

It's possible in a sense; it depends on what exactly you mean. Decorator syntax like this...

@dec
def foo():
    pass

is really just sugar for this:

def foo():
    pass
foo = dec(foo)

So there's nothing to stop you from using a decorator on a predefined function in the global namespace.

func = dec(func)

But the methods of built-in classes live in the namespace of that class, and that namespace can't be modified directly, as chepner has already pointed out. That's a good thing, because it ensures that objects of type str will behave as expected! However, you could subclass str and decorate the method that way. (The below works in Python 2; in Python 3, pass the output of filter to a list. super also may work a little differently; I'll post a Python 3 update in the future.)

>>> def remove_empty(fn):
...     def filtered(*args, **kwargs):
...         return filter(lambda x: x != '', fn(*args, **kwargs))
...     return filtered
... 
>>> class WeirdString(str):
...     @remove_empty
...     def split(self, *args, **kwargs):
...         return super(WeirdString, self).split(*args, **kwargs)
... 
>>> 'This decorator is unnecessary\n\n\n'.split('\n')
['This decorator is unnecessary', '', '', '']
>>> WeirdString('This decorator is unnecessary\n\n\n').split('\n')
['This decorator is unnecessary']

Or more directly (and so more in the spirit of decorator use):

>>> class WeirdString2(str):
...     split = remove_empty(str.split)
... 
>>> WeirdString2('This decorator is unnecessary\n\n\n').split('\n')
['This decorator is unnecessary']

In the case of this particular example, I'd prefer an explicit filter. But I can imagine, for example, a subclass of a built-in class that does some memoization or something like that.

It's also possible to make your subclass replace the existing `str` by replacing the built-in: `__builtins__.str = WeirdString`. This doesn't affect the `''` or `""` constructors, though, and should only be done if you really understand what you're doing. — Matthew Trevor, Sep 18 '12 at 12:57
Does this work in Python 3? Instead of `['This decorator is unnecessary']`, I get ``. — Adrian Keister, Jun 15 '18 at 17:58
@AdrianKeister, not quite, because in Python 3 the value returned by `filter` is an iterator instead of a list. So the call to `filter` would need to be wrapped in a call to `list`. I'll add a note -- thanks for the comment! — senderle, Jun 15 '18 at 18:21
I see. Well, if I try `list[filter(...` I get TypeError: 'type' object is not subscriptable. — Adrian Keister, Jun 15 '18 at 18:26

score 5 · Answer 2 · edited Sep 19 '22 at 11:42

I'm afraid the answer is no. Decorators are applied when the function is defined, and str.split is pre-defined. You might think you could do something explicit like

str.split = remove_empty(str.split)

but that is not permitted:

Traceback (most recent call last):
  File "tmp.py", line 8, in <module>
    str.split = remove_empty(str.split)
TypeError: can't set attributes of built-in/extension type 'str'

score 1 · Answer 3 · answered Sep 18 '12 at 11:51

1

Of course it is. Just write

remove_empty(lambda: some_string.split('\n'))()

answered Sep 18 '12 at 11:51

ecatmur

152,476
27
293
366

This just returns a new function that you would need to call again to filter `some_string`. It does not apply a decorator to `str.split`. – chepner Sep 18 '12 at 11:54
1

@chepner it applies a decorator to a function, and calls the result. – ecatmur Sep 18 '12 at 11:55
Sorry, parenthesis blindness. I didn't see the extra pair at the end to call the function. – chepner Sep 18 '12 at 11:57

Is it possible to apply my own decorators to builtin methods in Python?

3 Answers3

Linked