Saving Object State with Pickle (objects containing objects)

Question

I'm trying to figure out how to serialize an object with Pickle to a save file. My example is an object called World and this object has a list (named objects) of potentially hundreds of instantiated objects of different class types.

The problem is that Pickle won't let me serialize the items within the World.objects list because they aren't instantiated as attributes of World.

When I attempt to serialize with:

with open('gsave.pkl', 'wb') as output:
    pickle.dump(world.objects, output, pickle.DEFAULT_PROTOCOL)

I get the following error:

_pickle.PicklingError: Can't pickle <class 'world.LargeHealthPotion'>:
    attribute lookup LargeHealthPotion on world failed

So, my question is: what is an alternative way of storing the world.objects list items so that they are attributes of world rather than list items that don't get saved?

UPDATE I think my issue isn't where the objects are stored; but rather that the class LargeHealthPotion (and many others) are dynamically created within the World class by operations such as this:

def __constructor__(self, n, cl, d, c, h, l):
    # initialize super of class type
    super(self.__class__, self).__init__(name=n, classtype=cl, description=d, cost=c,
                                         hp=h, level=l)
    # create the object class dynamically, utilizing __constructor__ for __init__ method
    item = type(item_name,
                (eval("{}.{}".format(name,row[1].value)),),
                {'__init__':__constructor__})
    # add new object to the global _objects object to be used throughout the world
    self._objects[item_name] = item(obj_name, obj_classtype, obj_description, obj_cost,
                                    obj_hp, obj_level)

When this finishes, I will have a new object like <world.LargeHealthPotion object at 0x103690ac8>. I do this dynamically because I don't want to explicitly have to create hundreds of different types of classes for each different type of object in my world. Instead, I create the class dynamically while iterating over the item name (with it's stats) that I want to create.

This introduces a problem though, because when pickling, it can't find the static reference to the class in order to deconstruct, or reconstruct the object...so it fails.

What else can I do? (Besides creating literal class references for each, and every, type of object I'm going to instantiate into my world.)

Answer 1

Pickle does not pickle classes, it instead relies on references to classes which doesn't work if the class was dynamically generated. (this answer has appropriate exert and bolding from documentation)

So pickle assumes that if your object is from the class called world.LargeHealthPotion then it check that that name actually resolves to the class that it will be able to use when unpickling, if it doesn't then you won't be able to reinitialize the object since it doesn't know how to reference the class. There are a few ways of getting around this:

Define __reduce__ to reconstruct object

I'm not sure how to demo this method to you, I'd need much more information about your setup to suggest how to implement this but I can describe it:

First you'd make a function or classmethod that could recreate one object based on the arguments (probably take class name, instance variables etc.) Then define __reduce__ on the object base class that would return that function along with the arguments needed to pass to it when unpickling.

Put the dynamic classes in the global scope

This is the quick and dirty solution. Assuming the class names do not conflict with other things defined in the world module you could theoretically insert the classes into the global scope by doing globals()[item_name] = item_type, but I do not recommend this as long term solution since it is very bad practice.

Don't use dynamic classes

This is definitely the way to go in my opinion, instead of using the type constructor, just define your own class named something like ObjectType that:

• Is not a subclass of type so the instances would be pickle-able.
• When an instance is it called constructs a new game-object that has a reference to the object type.

So assuming you have a class called GameObject that takes cls=<ObjectType object> you could setup the ObjectType class something like this:

class ObjectType:
    def __init__(self, name, description):
        self.item_name = name
        self.base_item_description = description
        #other qualities common to all objects of this type

    def __call__(self, cost, level, hp):
        #other qualities that are specific to each item
        return GameObject(cls=self, cost=cost, level=level, hp=hp)

Here I am using the __call__ magic method so it uses the same notation as classes cls(params) to create instances, the cls=self would indicate to the (abstracted) GameObject constructor that the class (type) of GameObject is based on the ObjectType instance self. It doesn't have to be a keyword argument, but I'm not sure how else to make a coherent example code without knowing more about your program.