python: pickle can't to store local function (Can't pickle local object)

Question

In my class method I use count to generate unique IDs, which I want store within the class:

from collections import Counter, defaultdict
import pickle

class A:
   def __init__(self):
      self.models = {}

   def func1(self, data, names):
      ll = []
      for n, seq_list in enumerate(data):
          __counter = count(start=0)

          def _genid():
             return next(__counter)

          __ids = defaultdict(_genid)

          # Here we process seq_list and fill __ids
          # ...

          self.models[name] = [None, None, None, __ids]

   def save_models(self, obj, filename="models.bin"):
        try:
            with open(filename, 'wb') as f:
                pickle.dump(obj, f)
        except IOError as e:
            raise Exception("Failed to open file for writing: %s" % e)

...

data = [['A', 'BB', 'B', 'B', 'B'],
        [['C', 'D', 'E', 'B'], ['E', 'B'], ['C', 'D', 'E', 'B', 'B', 'F']],
        ['A', 'G'],
        [['AA', 'C', 'B', 'B'], ['F', 'D'], ['BB', 'E', 'E', 'F', 'B', 'A']]]
names = ['name1', 'name2', 'name3', 'name4'    

aclass = A()
aclass.func1(data, names)
aclass.save_models(aclass)

At this point save_models() generates error: AttributeError: Can't pickle local object 'A.func1.._genid'

So I have two questions: - Is there a way to have pickle save my class? - If not, if there is easier way to store ids in the class and be able to pickle it?

Answer 1

Option 1: Avoid _genid and use a simple integer variable for counting because pickle module doesn't support local functions.

Option 2: Use dill for pickling (replace import pickle with import dill as pickle)

Reference: Can Python pickle lambda functions?

Answer 2

The class cannot be pickled as is, but can be easily changed to make it pickle'able. Directly use __counter.__next__ instead of the _genid local wrapper:

__ids = defaultdict(__counter.__next__)

All of defaultdict, the count and its __next__ method can be pickled, allowing the resulting object to be pickled as well.