How to pickle a CookieJar?

Question

I have an object with a CookieJar that I want to pickle.

However as you all probably know, pickle chokes on objects that contain lock objects. And for some horrible reason, a CookieJar has a lock object.

from cPickle import dumps
from cookielib import CookieJar

class Person(object):
    def __init__(self, name):
        self.name = name
        self.cookies = CookieJar()

bob = Person("bob")
dumps(bob)

# Traceback (most recent call last):
#  File "<stdin>", line 1, in <module>
# cPickle.UnpickleableError: Cannot pickle <type 'thread.lock'> objects

How do I persist this?

The only solution I can think of is to use FileCookieJar.save and FileCookieJar.load to a stringIO object. But is there a better way?

Wow!! The question sounds like question for working in the kitchen. I am amazed the names used for the tools :) — shahkalpesh, Jun 21 '09 at 05:25
@Paul both Alex's and Anurag's solutions work. While Anurag's solution is hackish and faster, Alex's solution is more general and slower, so I will leave it up to the community to decide which is better. — Unknown, Jul 09 '09 at 22:38

Anurag Uniyal · Accepted Answer · 2012-02-23T19:14:32.853

9

Here is an attempt, by deriving a class from CookieJar, which override getstate/setstate used by pickle. I haven't used cookieJar, so don't know if it is usable but you can dump derived class

from cPickle import dumps
from cookielib import CookieJar
import threading

class MyCookieJar(CookieJar):
    def __getstate__(self):
        state = self.__dict__.copy()
        del state['_cookies_lock']
        return state

    def __setstate__(self, state):
        self.__dict__ = state
        self._cookies_lock = threading.RLock()

class Person(object):
    def __init__(self, name):
        self.name = name
        self.cookies = MyCookieJar()

bob = Person("bob")
print dumps(bob)

edited Feb 23 '12 at 19:14

answered Jun 21 '09 at 05:01

Anurag Uniyal

85,954
40
175
219

cons: it relies on internal logic of CookieJar – Anurag Uniyal Jun 21 '09 at 05:03
1

pros: it's simple and elegant! – Alex Jul 24 '09 at 20:00
1

for some definitions of 'elegant' =P. I do like this approach too, though. – Claudiu Aug 21 '09 at 09:53

score 7 · Answer 2 · answered Jun 21 '09 at 04:56

7

CookieJar is not particularly well-designed for persisting (that's what the FileCookieJar subclasses are mostly about!-), but you can iterate over a CookieJar instance to get all cookies (and persist a list of those, for example), and, to rebuild the jar given the cookies, use set_cookie on each. That's how I would set about persisting and unpersisting cookie jars, using the copy_reg method to register the appropriate functions if I needed to use them often.

answered Jun 21 '09 at 04:56

Alex Martelli

854,459
170
1,222
1,395

I feel this is a much better answer than the one that was accepted. It's possible that pickling is really what he needs, but it seems rather unlikely. The FileCookieJar subclasses are simpler, reliable, portable, and ready out of the box. – rspeed Apr 03 '13 at 03:21

How to pickle a CookieJar?

2 Answers2