3

I have a program in which I need to store a global variable into a file. I am doing this using the pickle module.

I have another thread(Daemon=False, from threading module) which sometimes changes the value of the global variable. The value is also modified in global scope(the main program).

I am dumping the value of the variable into a .pkl file every 5 seconds (using another thread from threading module).

But I found the following error when dump method was executed:

TypeError: can't pickle _thread.lock objects

Why is this happening? And what can I do to fix it?

Note: I have found some similar answers with multiprocessing module. But I need an answer for threading module.

Code:

def save_state():
    while True:
        global variable

        lastSession = open('lastSession.pkl', 'wb')

        # error occurs on this line
        pickle.dump(variable, lastSession)

        lastSession.close()          
        time.sleep(5)

state_thread = threading.Thread(target = save_state)
state_thread.setDaemon(False)
state_thread.start()

# variable is changed outside this function and also in another thread(not state_thread).
Bakuriu
  • 98,325
  • 22
  • 197
  • 231
  • 2
    you cannot pickle a lock object because it is tied to the curent state of the operating system / semaphores currently held by the other threads. – Jean-François Fabre Sep 02 '16 at 07:32
  • If you want to pickle a types which directly or indirectly references locks, you'll need to implement your own [`__getstate__`](https://docs.python.org/2/library/pickle.html#object.__getstate__) methods. It doesn't matter if the locks are released or not, they cannot be pickled. – Bi Rico Sep 02 '16 at 07:40
  • But when the target function is terminated, shouldn't the locks be released since there is only one thread remaining? Also if you can provide a sample code, that would really help –  Sep 02 '16 at 08:11
  • To discuss your code, we'd really need to see it. Can you show us the lock in question? Is it an attribute of some object that you're pickling, or are you trying to pickle it directly? – Blckknght Sep 02 '16 at 10:19

1 Answers1

0

As others have mentioned, you cannot pickle "volatile" entities (threads, connections, synchonization primitives etc.) because they don't make sense as persistent data.

Looks like what you're trying to do is be saving session variables for later continuation of said session. For that task, there's nothing you can do to save objects that, well, cannot be saved due to their nature.

The simplest solution is just to ignore them. Replacing them with some "stubs" that would yield an error anytime you do anything with them makes little sense, as a missing variable yields an error anyway (it won't if it was masking a global variable but that is a questionable practice in itself).

Alternatively, you can set up these objects anew when restoring a session. But such a logic is by necessity task-specific.

Finally, e.g. IPython already has session saving/restoration logic, so you probably don't need to reinvent the wheel at all.

Community
  • 1
  • 1
ivan_pozdeev
  • 33,874
  • 19
  • 107
  • 152
  • Can you be more specific with the session logic in ipython? Also Can I store these volatile variables in a database ? –  Sep 03 '16 at 11:34
  • 1st: done, 2nd: did you get the gist of serialization and the "don't make sense as persistent data" part? – ivan_pozdeev Sep 03 '16 at 15:59