Fix mutable default arguments via metaclass

Question

The issue with mutable argument default values is pretty well known in Python. Basically mutable default values are assigned once at define time and can then be modified within the function body which might come as a surprise.

---

Today at work we were thinking about different ways to deal with this (next to testing against None which apparently is the right way...) and I came up with a Metaclass solution that you can find here or down below (it's a few lines so the gist might be more readable).

It basically works like this:

1. For each function obj. in the attributes dict.
2. Introspect function for mutable default args.
3. If mutable default args. are found, replace the function with a decorated function
4. The decorated function was created with a closure that registered the default arg. name and initial default value
5. On each function call, check if a kwarg. by the registered name was given and if it was NOT given, re-instanciate the initial value to create a shallow copy and add it to the kwargs before execution.

---

The problem now is that this approach works great for list and dict objects, but it somehow fails for other mutable default values like set() or bytearray(). Any ideas why?
Feel free to test this code. The only non-standard dep. is six (pip install six) so it works in Py2 and 3.

# -*- coding: utf-8 -*-
import inspect
import types
from functools import wraps
from collections import(
    MutableMapping,
    MutableSequence,
    MutableSet
)

from six import with_metaclass  # for py2/3 compatibility | pip install six


def mutable_to_immutable_kwargs(names_to_defaults):
    """Decorator to return function that replaces default values for registered
    names with a new instance of default value.
    """
    def closure(func):
        @wraps(func)
        def wrapped_func(*args, **kwargs):

            set_kwarg_names = set(kwargs)
            set_registered_kwarg_names = set(names_to_defaults)
            defaults_to_replace = set_registered_kwarg_names - set_kwarg_names

            for name in defaults_to_replace:
                define_time_object = names_to_defaults[name]
                kwargs[name] = type(define_time_object)(define_time_object)

            return func(*args, **kwargs)
        return wrapped_func
    return closure


class ImmutableDefaultArguments(type):
    """Search through the attrs. dict for functions with mutable default args.
    and replace matching attr. names with a function object from the above
    decorator.
    """

    def __new__(meta, name, bases, attrs):
        mutable_types = (MutableMapping,MutableSequence, MutableSet)

        for function_name, obj in list(attrs.items()):
            # is it a function ?
            if(isinstance(obj, types.FunctionType) is False):
                continue

            function_object = obj
            arg_specs = inspect.getargspec(function_object)
            arg_names = arg_specs.args
            arg_defaults = arg_specs.defaults

            # function contains names and defaults?
            if (None in (arg_names, arg_defaults)):
                continue

            # exclude self and pos. args.
            names_to_defaults = zip(reversed(arg_defaults), reversed(arg_names))

            # sort out mutable defaults and their arg. names
            mutable_names_to_defaults = {}
            for arg_default, arg_name in names_to_defaults:
                if(isinstance(arg_default, mutable_types)):
                    mutable_names_to_defaults[arg_name] = arg_default

            # did we have any args with mutable defaults ?
            if(bool(mutable_names_to_defaults) is False):
                continue

            # replace original function with decorated function
            attrs[function_name] = mutable_to_immutable_kwargs(mutable_names_to_defaults)(function_object)


        return super(ImmutableDefaultArguments, meta).__new__(meta, name, bases, attrs)


class ImmutableDefaultArgumentsBase(with_metaclass(ImmutableDefaultArguments,
                                                   object)):
    """Py2/3 compatible base class created with ImmutableDefaultArguments
    metaclass through six.
    """
    pass


class MutableDefaultArgumentsObject(object):
    """Mutable default arguments of all functions should STAY mutable."""

    def function_a(self, mutable_default_arg=set()):
        print("function_b", mutable_default_arg, id(mutable_default_arg))


class ImmutableDefaultArgumentsObject(ImmutableDefaultArgumentsBase):
    """Mutable default arguments of all functions should become IMMUTABLE.
    through re-instanciation in decorated function."""

    def function_a(self, mutable_default_arg=set()):
        """REPLACE DEFAULT ARGUMENT 'set()' WITH [] AND IT WORKS...!?"""
        print("function_b", mutable_default_arg, id(mutable_default_arg))


if(__name__ == "__main__"):

    # test it
    count = 5

    print('mutable default args. remain with same id on each call')
    mutable_default_args = MutableDefaultArgumentsObject()
    for index in range(count):
        mutable_default_args.function_a()

    print('mutable default args. should have new idea on each call')
    immutable_default_args = ImmutableDefaultArgumentsObject()
    for index in range(count):
        immutable_default_args.function_a()

Answer 1

Your code as it stands is actually doing what you expect. It is passing a new copy of the default to the function when called. However, since you do nothing with this new value it is garbage collected and the memory is free for immediate reallocation on your very next call.

Thus, you keep getting the same id().

The fact that the id() for two objects at different points in time is the same does not indicate they are the same object.

To see this effect, alter your function so it does something with the value that will increase its reference count, such as:

class ImmutableDefaultArgumentsObject(ImmutableDefaultArgumentsBase):
    cache = []
    def function_a(self, mutable_default_arg=set()):
        print("function_b", mutable_default_arg, id(mutable_default_arg))
        self.cache.append(mutable_default_arg)

Now running your code will provide:

function_b set() 4362897448
function_b set() 4362896776
function_b set() 4362898344
function_b set() 4362899240
function_b set() 4362897672