Why does deepcopy fail when copying a complex object

Question

How can I replicate a complex object so that I can add new members to it? When I try to use deepcopy, it fails with "TypeError: cannot serialize ..."

The original problem is that I want to add some member variables to an existing object but can't because doing so results in "AttributeError: Object is fixed"

So the idea is create a full copy of the original object in a new class with the added members.

orig_obj = SomeSqlObject.get_root() # contents unclear, complex

class orig_expanded():
    def __init__(self, replicable_object):
        self.__dict__ = copy.deepcopy(replicable_object.__dict__)

        self.added_member1 = None
        self.added_list    = []

expanded_thing = orig_expanded(orig_obj)

But I get:

TypeError: cannot serialize '_io.TextIOWrapper' object

Followup answer to comment, "What is SomeSqlObject?" Perhaps my name is wrong... actual name obfuscated for the company. It is a method that returns an object that represents the base of a tree (of some kind) That tree is defined

class SomeSqlObject(ParentRegisterSet):
    """
    Implements the functionality of the Device "root" Register Set.

    """
    def __init__(self, db, v1, dg, ui):
        self.__db__ = db
        self.__dg__ = dg
        self.__ui__ = ui
        SomeSqlObject.__init__(self, v1, None)

        # note:  this class is now locked

Answer 1

copy.deepcopy's behavior for classes that don't provide direct support (by defining __deepcopy__) is to pickle then unpickle the object to ensure a new instance is created. io.TextIOWrapper (which is a wrapper than converts binary file-like objects to text file-like objects) can't be serialized (because it assumes it may have external/run time state, e.g. a file descriptor that has a specific position in a file that may not be available when it's later deserialized).

The error comes because the object you're copying contains io.TextIOWrapper, and the serialization fails.

If the shared state is okay, you might limit yourself to a shallow copy, or use a composition based wrapper (based on __getattr__) to access the underlying object through the wrapper object semi-seamlessly (aside from those pesky special methods), or you might try to individually deepcopy the values from the dictionary and just ignore the ones you can't copy, e.g.:

for attr, value in vars(replicable_object).items():
    try:
        setattr(self, attr, copy.deepcopy(value))
    except Exception:
        pass
        # Alternatively, copy reference when copy impossible:
        #setattr(self, attr, value)

and just hope that the stuff you can't copy isn't too important.

Answer 2

TypeError: cannot serialize '_io.TextIOWrapper' object

This exception means that somewhere, somehow your object is linked to a file object, a socket or something like that.

TextIOWrapper is the class that wraps file descriptors and lets you read/write unicode strings.

And, as you can see, TextIOWrapper cannot be copied.

Answer 3

My guess is what you really want is a proxy class, one example from google: http://python-3-patterns-idioms-test.readthedocs.org/en/latest/Fronting.html

You will initialize your proxy class from the object to wrap; attributes that the proxy class knows are handled locally; attributes that the proxy class doesn't know about are passed through to the wrapped object.

(normally, you'd subclass if you're creating these objects yourself... sounds like that is not an option here...)

Answer 4

Ok Found The answer.

The original error seen when trying to do a setattr() was AttributeError: Object is fixed. That was an error from code in the original SomeSqlObject for a custom __setatter__() that was looking checking for bit _attr_lock and preventing adding members to the object. Once I de-asserted this lock, I was able to add members easily.

The original problem is that I have a number of class members (call them id's) of the form id0, id1, id3, id2, etc. Each of them is a complex object also. But, the better way from the code-user's point of view is to use a list-type member id[#] to access them. So, I needed to add on a list-type member id[] and make sure each successive element points to the same objects as pointed to by id0, id1, ie. id[0], id[1], etc.

So, my final code to take a complex object and add a list-type member, is.

# Below we ADD ON to the existing SqlObject with a list-type member
# and populate the list with proper pointers to each register/register-
# set. This is PFM!

id_name_re = "\W*id\D*(\d+)"

# SqlObject has a feature to block adding attributes.... This will override
self.regs._attr_lock = None

# add a list-type placeholder in the SqlObj for a id array
setattr(self.regs, 'id', [])

# now get a list of all SqlObject  members called id<#> like:
#  ['id13', 'id12', 'id11', 'id10', 'id9', 'id8', 'id14 ...
id_list = [id_member  for id_member in self.regs.__dict__ if re.match(id_name_re, id_member)]

# Sort the list since we need to place them in the new id[] list
# sequentially
id_list = sorted(id_list, key=h.natural_sort_key)

# now go through the list and create a new list element and populate
# it with the SqlObject  goofy-name which is not in a list-type format
for id_member in id_list:
    offset = int(re.match(id_name_re, id_member).group(1))

    # this is NEEDED!. It causes the SqlObject  to rescan and keep
    # everything updated ('_' is kinda like /dev/null)
    _ = eval("self.regs.id%d" % (offset))

    self.regs.id.append(self.regs.__dict__[id_member].__getitem__.__self__)