How do you pass options to deepcopy?

Question

In this question, Anthony Hatchkins gives a default implementation of deepcopy based on dict-copying code that Python falls back to:

def __deepcopy__(self, memo):
    cls = self.__class__
    result = cls.__new__(cls)
    memo[id(self)] = result
    for k, v in self.__dict__.items():
        setattr(result, k, deepcopy(v, memo))
    return result

I would like a default implementation based on pickling and unpickling, which Python chooses before falling back to dict-copying.

Here is my attempt, which did not work:

def __deepcopy__(self, memo):
    new, args, state = self.__reduce__()
    result = new(*args)
    if state:
        result.__setstate__(state)
    memo[id(self)] = result
    return result

Once I have such a method, I can make a version with additional options about what is copied and how copying is done.

The existence of reduce and and setstate are guaranteed by a base class that defines:

@staticmethod
def kwargs_new(cls, new_kwargs, *new_args):
    """
    Override this method to use a different new.
    """
    retval = cls.__new__(cls, *new_args, **new_kwargs)
    retval.__init__(*new_args, **new_kwargs)
    return retval


"""
Define default getstate and setstate for use in coöperative inheritance.
"""
def __getstate__(self):
    return {}

def __setstate__(self, state):
    self.__dict__.update(state)

def __getnewargs_ex__(self):
    return ((), {})

def __reduce__(self):
    """
    Reimplement __reduce__ so that it calls __getnewargs_ex__
    regardless of the Python version.

    It will pass the keyword arguments to object.__new__.
    It also exposes a kwargs_new static method that can be overridden for
    use by __reduce__.
    """

    new_args, new_kwargs = self.__getnewargs_ex__()
    state = self.__getstate__()

    return (type(self).kwargs_new,
            (type(self), new_kwargs,) + tuple(new_args),
            state)

Why do you want a pickle-based solution? I don't see the advantage. Also, getting the pickler memo and the deepcopy memo to work nicely together could be a problem. — user2357112, Jun 18 '14 at 08:56
@user2357112: Because then I get pickling for free. I can use the same code to implement "save" and "load". — Neil G, Jun 18 '14 at 08:59
You get deepcopying for free by implementing the pickle protocol, e.g. by overriding `__getstate__` and `__setstate__`. [This question](http://stackoverflow.com/q/22388854/2096752) might be relevant. — shx2, Sep 17 '14 at 21:58
@shx2: Yes, that's what I ended up doing. The base class still needs the above code to deal with getnewargs_ex, etc. — Neil G, Sep 17 '14 at 22:11

Neil G · Answer 1 · 2014-06-18T10:10:05.503

The state passed to setstate needs to be copied as does the argument list:

def __deepcopy__(self, memo):
    """
    Reimplement __deepcopy__ so that
    * it supports keyword arguments to reduce.
    """
    kwargs = memo.get('deepcopy kwargs', {})
    new, args, state = self.__reduce__(**kwargs)
    args_copy = copy.deepcopy(args, memo)
    result = new(*args_copy)
    memo[id(self)] = result
    if state:
        state_copy = copy.deepcopy(state, memo)
        result.__setstate__(state_copy)
    return result

This version of deepcopy has been modified to use a special memo to pass keyword arguments to reduce.

score 0 · Answer 2 · answered Jun 18 '14 at 10:10

Another option is to pass keyword arguments to reduce by having reduce inspect the stack frame:

def f():
    import inspect
    for frame_tuple in inspect.stack():
        if 'x' in frame_tuple[0].f_locals:
            x = frame_tuple[0].f_locals['x']
    print(x)

def g():
    x = 5

    f()

g()

How do you pass options to deepcopy?

2 Answers2