35

Is there a possibility to create real copies of python functions? The most obvious choice was http://docs.python.org/2/library/copy.html but there I read:

It does “copy” functions and classes (shallow and deeply), by returning the original object unchanged;

I need a real copy, because I might change some attributes of the function.

Update:

I'm aware of all the possibilities which are mentioned in the comments. My use case is based on meta programming where I construct classes out of some declarative specifications. Complete details would be too long for SO, but basically I have a function like

def do_something_usefull(self,arg):
    self.do_work()

I will add this method to various classes. Thoses classes can be completly unrelated. Using mixin classes is not an option: I will have many such functions and would end up adding a base class for each function. My current "workaround" would be to wrap this function in a "factory" like this:

def create_do_something():
    def do_something_usefull(self,arg):
        self.do_work()

That way I always get a new do_something_useful function, but I have to wrap all my functions like this.

You can trust me, that I'm aware, that this is no "normal" OO programming. I know how to solve something like that "normally". But this is a dynamic code generator and I would like to keep everything as lightweight and simple as possible. And as python functions are quite normal objects, I don't think it's too strange to ask how to copy them!?

Achim
  • 15,415
  • 15
  • 80
  • 144
  • 1
    Use `copy.deepcopy` if you really must. But I think there's a deeper design flaw somewhere. Why do you want to copy functions in the first place? – inspectorG4dget Nov 21 '12 at 22:27
  • 2
    Can you help us understand a bit about what you want to do -- your question is a little strange/vague. In the thousands of lines of python I've written, I've never copied a function. – jfaller Nov 21 '12 at 22:28
  • 1
    If you need to maintain state, why not use classes and instances? Why functions that need to be *copied* somehow? – Praveen Gollakota Nov 21 '12 at 22:29
  • 3
    If you're changing attributes on a function you're probably better off wrapping the function in an object then changing attributes on the object. It will make your code cleaner. If you add a `__call__` method to your object it can be called just like a function. – Peter Graham Nov 21 '12 at 22:30
  • The only application of this that I can think of off the top of my head is for use in evolutionary algorithms (genetic programming). And even there, you might want to go with a different approach - enoding and decoding, rather than function copying – inspectorG4dget Nov 21 '12 at 22:31
  • Jeez... I had never thought about this... BTW, deepcopy returns, indeed, the same object (original function and "copied" function are both the same memory address) – Savir Nov 21 '12 at 22:34
  • 1
    If you want the same function with slightly different parameters, most likely, you either want to use closures or `functools.partial`. It's unlikely that you actually need to copy the code object. – Joel Cornett Nov 21 '12 at 22:40
  • If you want to reuse a function you don't need to copy it. Just reference the one function object from everywhere you need it. You only need to copy the function object if you're going to change its state. Storing state on a function object (or an unbound method?) is unusual and likely to cause confusion. It's far clearer to store state somewhere else. – Peter Graham Nov 21 '12 at 23:04
  • 5
    I don't see why treating function like any other object should be a problem, in python. I do have the same need: decorated functions are passed to internal **and external** code which expect a function object. Copying a function in order to change its decoration is thus useful and working around with objects that look-a-like function but are not is just overly complicated. – Juh_ Aug 07 '13 at 11:32
  • 1
    Overly complicated and slow. Function call overhead in python is huge, so "using closures" isn't really a solution. – DylanYoung Nov 30 '20 at 18:03
  • A function factory is decent alternate solution though (if you have control of the code where the function is defined). – DylanYoung Nov 30 '20 at 18:05

1 Answers1

50

In Python3:

import types
import functools

def copy_func(f):
    """Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)"""
    g = types.FunctionType(f.__code__, f.__globals__, name=f.__name__,
                           argdefs=f.__defaults__,
                           closure=f.__closure__)
    g = functools.update_wrapper(g, f)
    g.__kwdefaults__ = f.__kwdefaults__
    return g

def f(arg1, arg2, arg3, kwarg1="FOO", *args, kwarg2="BAR", kwarg3="BAZ"):
    return (arg1, arg2, arg3, args, kwarg1, kwarg2, kwarg3)
f.cache = [1,2,3]
g = copy_func(f)

print(f(1,2,3,4,5))
print(g(1,2,3,4,5))
print(g.cache)
assert f is not g

yields

(1, 2, 3, (5,), 4, 'BAR', 'BAZ')
(1, 2, 3, (5,), 4, 'BAR', 'BAZ')
[1, 2, 3]

In Python2:

import types
import functools
def copy_func(f):
    """Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)"""
    g = types.FunctionType(f.func_code, f.func_globals, name=f.func_name,
                           argdefs=f.func_defaults,
                           closure=f.func_closure)
    g = functools.update_wrapper(g, f)
    return g

def f(x, y=2):
    return x,y
f.cache = [1,2,3]
g = copy_func(f)

print(f(1))
print(g(1))
print(g.cache)
assert f is not g

yields

(1, 2)
(1, 2)
[1, 2, 3]
unutbu
  • 842,883
  • 184
  • 1,785
  • 1,677
  • 5
    The function dictionary should also be copied: `g.__dict__.update(f.__dict__)` – Juh_ Aug 07 '13 at 11:36
  • Is `func_code` (and friends) the Python 2 equivalent of `__code__` (and friends)? – weberc2 Apr 18 '16 at 20:27
  • @weberc: Yes, that's right. I've update the answer for Python3. – unutbu Apr 18 '16 at 20:56
  • The answer is incomplete. You should copy `f.__kwdefaults__` too. – Elazar Jun 17 '16 at 16:22
  • 1
    [`functools.update_wrapper`](https://docs.python.org/3/library/functools.html?highlight=functools.update_wrapper#functools.update_wrapper) would update all the metadata other then `__kwdefaults__` like `__doc__` and `__module__` – Tadhg McDonald-Jensen Jun 17 '16 at 19:10
  • Note that this is a shallow copy. If you want to do deepcopy you should call `copy.deepcopy` on all the writable attributes. – Elazar Jun 18 '16 at 22:00
  • Here's an [example](https://gist.github.com/elazarg/99e6d1450c3c85aa74317c1b01767d86) of why it is useful – Elazar Jun 18 '16 at 22:03
  • @Elazar. Here's another: https://stackoverflow.com/q/49076566/2988730 – Mad Physicist Mar 02 '18 at 20:46
  • For consistency, I think you may want to do `copy.copy(f.__kwdefaults__)`. – Mad Physicist Mar 02 '18 at 21:34
  • 1
    Currently, this answer is based on Glenn Maynard's answer to a similar question. Under that same question, there is [an updated answer by Aaron Hall](https://stackoverflow.com/a/30714299/10155767) which is very similar to this answer. Aaron updates the new __dict__ based on the old, while this answer uses [update_wrapper](https://github.com/python/cpython/blob/99e9c36b29a194d59ee4b0d8b82a72f5b0d409dd/Lib/functools.py#L59) to update the attributes individually. I am wondering (in the case of Python 3) whether or not the outcome is always the same. – Ben Mares Feb 19 '19 at 10:57
  • @Elazar Do you mean `getattr(wrapper, attr).update(copy.deepcopy(getattr(wrapped, attr, {})))` inside [update_wrapper](https://github.com/python/cpython/blob/99e9c36b29a194d59ee4b0d8b82a72f5b0d409dd/Lib/functools.py#L60) for deepcopy? – Ben Mares Feb 19 '19 at 11:01
  • 1
    @BenMares: With `g = functools.update_wrapper(g, f)`, you get `g.__wrapped__` which equals `f`. If you use `fn.__dict__.update(f.__dict__)` you do not get the `__wrapped__` attribute. I'm not aware of any other difference due to `update_wrapper`. – unutbu Feb 19 '19 at 13:06
  • 1
    @BenMares: A more dramatic difference is made by the [inclusion of `g.__kwdefaults__ = f.__kwdefaults__`](https://stackoverflow.com/questions/13503079/how-to-create-a-copy-of-a-python-function/13503277?noredirect=1#comment63227017_13503277). With this, `g(1,2,3)` returns `(1, 2, 3, (), 'FOO', 'BAR', 'BAZ')`. Without it, `g(1,2,3)` raises `TypeError: f() missing 3 required positional arguments: 'arg1', 'arg2', and 'arg3'`. – unutbu Feb 19 '19 at 13:06
  • It would make sense to have such a function in a library, until there is a built-in version. It could fit in `makefun` : https://github.com/smarie/python-makefun/issues/74 , if anyone is interested to propose a PR. – smarie Oct 08 '21 at 07:42