37

I would like to make a deepcopy of a function in Python. The copy module is not helpful, according to the documentation, which says:

This module does not copy types like module, method, stack trace, stack frame, file, socket, window, array, or any similar types. It does “copy” functions and classes (shallow and deeply), by returning the original object unchanged; this is compatible with the way these are treated by the pickle module.

My goal is to have two functions with the same implementation but with different docstrings.

def A():
    """A"""
    pass

B = make_a_deepcopy_of(A)
B.__doc__ = """B"""

So how can this be done?

Tom
  • 403
  • 1
  • 4
  • 4

9 Answers9

32

The FunctionType constructor is used to make a deep copy of a function.

import types
def copy_func(f, name=None):
    return types.FunctionType(f.func_code, f.func_globals, name or f.func_name,
        f.func_defaults, f.func_closure)

def A():
    """A"""
    pass
B = copy_func(A, "B")
B.__doc__ = """B"""
Glenn Maynard
  • 55,829
  • 10
  • 121
  • 131
  • 1
    @GlennMaynard : this can’t work for built‑in functions http://stackoverflow.com/q/39353158/2284570 – user2284570 Sep 06 '16 at 16:21
  • 3
    Only works in python 2. See [Aaron Hall's](https://stackoverflow.com/a/30714299/425458) post below for an answer that's py 2/3 compatible. – tel Feb 11 '18 at 03:37
29

My goal is to have two functions with the same implementation but with different docstrings.

Most users will do this, say the original function is in old_module.py:

def implementation(arg1, arg2): 
    """this is a killer function"""

and in new_module.py

from old_module import implementation as _implementation

def implementation(arg1, arg2):
    """a different docstring"""
    return _implementation(arg1, arg2)

This is the most straightforward way to reuse functionality. It is easy to read and understand the intent.

Nevertheless, perhaps you have a good reason for your main question:

How can I make a deepcopy of a function in Python?

To keep this compatible with Python 2 and 3, I recommend using the function's special __dunder__ attributes. For example:

import types

def copy_func(f, name=None):
    '''
    return a function with same code, globals, defaults, closure, and 
    name (or provide a new name)
    '''
    fn = types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
        f.__defaults__, f.__closure__)
    # in case f was given attrs (note this dict is a shallow copy):
    fn.__dict__.update(f.__dict__) 
    return fn

And here's an example usage:

def main():
    from logging import getLogger as _getLogger # pyflakes:ignore, must copy
    getLogger = copy_func(_getLogger)
    getLogger.__doc__ += '\n    This function is from the Std Lib logging module.\n    '
    assert getLogger.__doc__ is not _getLogger.__doc__
    assert getLogger.__doc__ != _getLogger.__doc__

A commenter says:

This can’t work for built‑in functions

Well I wouldn't do this for a built-in function. I have very little reason to do this for functions written in pure Python, and my suspicion is that if you are doing this, you're probably doing something very wrong (though I could be wrong here).

If you want a function that does what a builtin function does, and reuses the implementation, like a copy would, then you should wrap the function with another function, e.g.:

_sum = sum
def sum(iterable, start=0):
    """sum function that works like the regular sum function, but noisy"""
    print('calling the sum function')
    return _sum(iterable, start)
    
Community
  • 1
  • 1
Russia Must Remove Putin
  • 374,368
  • 89
  • 403
  • 331
  • 1
    This can’t work for built‑in functions http://stackoverflow.com/q/39353158/2284570 – user2284570 Sep 06 '16 at 16:22
  • py 2/3 compatibility for the win! – tel Feb 11 '18 at 03:35
  • 1
    [This solution](https://stackoverflow.com/a/13503277/10155767) of a duplicate problem is very similar, but also takes into account a few further subtleties such as `__kwdefaults__`. – Ben Mares Feb 19 '19 at 13:25
  • 2
    Maybe this technique isn't well motivated for docstrings, but I'm glad you wrote the tutorial because I really needed it! I needed a copy of a suite of functions, which call each other, to be copied to a new (dynamic) module and call each others' copies, rather than the originals. This meant replacing their `__globals__` to point to the new module. The reason for all of this was to Numba-compile all of the copied functions without affecting the original functions, since they still need to be accessed in pure object mode. So thanks! – Jim Pivarski Mar 18 '21 at 22:36
8
from functools import partial

def a():
    """Returns 1"""
    return 1

b = partial(a)
b.__doc__ = """Returns 1, OR DOES IT!"""

print help(a)
print help(b)

Wrap it as a partial?

Jakob Bowyer
  • 33,878
  • 8
  • 76
  • 91
2

The others answers do not allow for serialization with pickle. Here a code that I am using to clone a function and allow for serialization for python3:

import pickle
import dill
import types

def foo():
    print ('a')


oldCode=foo.__code__

name='IAmFooCopied'

newCode= types.CodeType(
        oldCode.co_argcount,             #   integer
        oldCode.co_kwonlyargcount,       #   integer
        oldCode.co_nlocals,              #   integer
        oldCode.co_stacksize,            #   integer
        oldCode.co_flags,                #   integer
        oldCode.co_code,                 #   bytes
        oldCode.co_consts,               #   tuple
        oldCode.co_names,                #   tuple
        oldCode.co_varnames,             #   tuple
        oldCode.co_filename,             #   string
        name,                  #   string
        oldCode.co_firstlineno,          #   integer
        oldCode.co_lnotab,               #   bytes
        oldCode.co_freevars,             #   tuple
        oldCode.co_cellvars              #   tuple
        )

IAmFooCopied=types.FunctionType(newCode, foo.__globals__, name,foo.__defaults__ , foo.__closure__)
IAmFooCopied.__qualname__= name
print ( 'printing foo and the copy', IAmFooCopied, foo )
print ( 'dill output: ', dill.dumps(IAmFooCopied ))
print ( 'pickle Output: ', pickle.dumps (IAmFooCopied) )

Output:

printing foo and the copy <function IAmFooCopied at 0x7f8a6a8159d8> <function foo at 0x7f8a6b5f5268>
dill output:  b'\x80\x03cdill._dill\n_create_function\nq\x00(cdill._dill\n_load_type\nq\x01X\x08\x00\x00\x00CodeTypeq\x02\x85q\x03Rq\x04(K\x00K\x00K\x00K\x02KCC\x0ct\x00d\x01\x83\x01\x01\x00d\x00S\x00q\x05NX\x01\x00\x00\x00aq\x06\x86q\x07X\x05\x00\x00\x00printq\x08\x85q\t)X\x10\x00\x00\x00testCloneFunc.pyq\nX\x0c\x00\x00\x00IAmFooCopiedq\x0bK\x05C\x02\x00\x01q\x0c))tq\rRq\x0ec__builtin__\n__main__\nh\x0bNN}q\x0ftq\x10Rq\x11.'
pickle Output:  b'\x80\x03c__main__\nIAmFooCopied\nq\x00.'

You may encounter problem with the qualname attribute if you try this snippet with class methods (I think pickle should fail to find your function). I never tried it, however it should be easily fixable. Just check the doc about qualname

ninjaconcombre
  • 456
  • 4
  • 15
  • 1
    somehow people overlooked this piece – oldpride Aug 09 '20 at 23:19
  • 1
    **Brilliant.** This is undeniably the optimally efficient solution to function copying both here [and elsewhere](https://stackoverflow.com/questions/13503079/how-to-create-a-copy-of-a-python-function). Sadly, it lacks upvotes. Sadly, it's also incomplete. Why? Because it fails to copy the `__kwdefaults__`, `__module__`, `__qualname__`, `__doc__`, and `__annotations__` attributes from the old to new function. Fortunately, *all* of those attributes can be trivially copied by just calling `functools.update_wrapper(IAmFooCopied, foo, functools.WRAPPER_ASSIGNMENTS + ('__kwdefaults__',))`. **Yup.** – Cecil Curry Feb 15 '21 at 05:19
  • **Addendum:** [the new `CodeType.replace()` method introduced in Python 3.8](https://docs.python.org/3/library/types.html#types.CodeType.replace) is strongly preferable to reconstructing a new `CodeType` object from scratch. Why? Because future-proofing. The `CodeType` constructor frequently changes between Python versions, which then breaks everything. – Cecil Curry Apr 08 '21 at 21:32
2
def A():
    """A"""
    pass

def B():
    """B"""
    return A()
Pushpak Dagade
  • 6,280
  • 7
  • 28
  • 41
  • It works, but is there something nicer...which doesn't increase the function call overhead? I literally just want a copy. – Tom Jun 29 '11 at 21:51
  • I guess, even if there would be a way, it won't be 'nicer' – Pushpak Dagade Jun 29 '11 at 21:53
  • I agree it won't be nicer, but since I am calling this function in for loops nested a few levels down, I went with the factory method. – Tom Jun 29 '11 at 22:00
1

It's quite easy to do using lambda and rest parameters:

def my_copy(f): 
        # Create a lambda that mimics f
        g = lambda *args: f(*args)
        # Add any properties of f
        t = list(filter(lambda prop: not ("__" in prop),dir(f)))
        i = 0
        while i < len(t):
            setattr(g,t[i],getattr(f,t[i]))
            i += 1
        return g
        
# Test
def sqr(x): return x*x
sqr.foo = 500

sqr_copy = my_copy(sqr)
print(sqr_copy(5)) # -> 25
print(sqr_copy(6)) # -> 36
print(sqr_copy.foo) # -> 500
print(sqr_copy == sqr) # -> False

Try it online!

Nirvana
  • 405
  • 3
  • 15
0

put it in a function:

def makefunc( docstring ):
    def f():
        pass
    f.__doc__ = docstring
    return f

f = makefunc('I am f')
g = makefunc("I am f too")
Jochen Ritzel
  • 104,512
  • 31
  • 200
  • 194
  • Alright, I guess I am going to cave in and accept this. I was hoping there was something which didn't rely or wrapping or a construction, but I suppose it ultimately comes down to that at some level. – Tom Jun 29 '11 at 21:56
  • @Tom did you not look at my example? It shows basically the syntax you want. – Jakob Bowyer Jun 29 '11 at 22:04
0

Adjusted for python3

import types
def copy_func(f, name=None):
    return types.FunctionType(f.__code__, f.__globals__, name or f.__name__,
        f.__defaults__, f.__closure__)
def func1(x):
  return 2*x
func2=copy_func(func1)
print(func2(7))
hmghaly
  • 1,411
  • 3
  • 29
  • 47
0

I've implemented a general-purpose function copy in haggis, a library which I wrote and maintain (available with pip but probably not conda). haggis.objects.copy_func makes a copy on which you can not only reassign the __doc__ attribute, but also modify the module and __globals__ attributes effectively.

from haggis.objects import copy_func

def a(*args, **kwargs):
    """A docstring"""

a2 = copy_func(a)
a2.__doc__ = """Another docstring"""
>>> a == a2
False
>>> a.__code__ == a2.__code__
True
>>> a.__doc__
'A docstring'
>>> a2.__doc__
'Another docstring'
Mad Physicist
  • 107,652
  • 25
  • 181
  • 264