0

I'm trying to write some data to a cache of which some has been retrieved from a postgres db using psycopg2 and sqlalchemy. Some of them are NumericRange instances. Unfortunately, when I try to pickle the objects before writing them to the Redis cache, they raise an exception.

lib/python2.7/copy_reg.py", line 77, in _reduce_ex
    raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickled

Is it possible to change pickle, so it can deal with NumericRange objects? Note cpickle is used for the serialization.

orange
  • 7,755
  • 14
  • 75
  • 139
  • My psychic powers tell me that this is a bug in SQLAlchemy, similar to [this bug in Django](https://code.djangoproject.com/ticket/25501). – Kevin Aug 30 '16 at 02:51
  • Well, more `psycopg2`, but that awareness doesn't really help me... – orange Aug 30 '16 at 02:54
  • 1
    The problem with Django is that they were calling `pickle.dumps(value)` instead of `pickle.dumps(value, -1)`. You need the latter. But if this pickling is happening deep inside SQLAlchemy, then you need to talk to the developers of SQLAlchemy. – Kevin Aug 30 '16 at 02:57
  • Actually, that helps. I just tried `protocol=2` which apparently is for Python 2.x (https://docs.python.org/3/library/pickle.html#data-stream-format) and it seems to work. The pickling is happening in a Flask extension which I can override. Thanks. – orange Aug 30 '16 at 03:01
  • 1
    An alternative (for future readers who can't alter the pickle protocol) is to register a new type on the psycopg2 connection itself that will convert a numeric range to a child class that can be pickled. See http://initd.org/psycopg/docs/extensions.html#database-types-casting-functions – donkopotamus Aug 30 '16 at 03:12
  • @donkopotamus Interesting, thanks for the comment. – orange Aug 30 '16 at 03:14

0 Answers0