multi threads modify a global list in python

Question

i want to add an item into a global list every 2 seconds in one thread, and save the list into database before empty it every 3 seconds in another thread.

i create two local varibles to monitor the total added items and total saveditems, they should be equal every 6 senconds,but it is not. here is my code:

import datetime
import psutil,os,time
from threading import *
class AddToList(Thread):
    totalAdded=0
    def run(self):
        lock=RLock()
        lock.acquire()
        while True:
            entryList.append("AddToList at "+str(datetime.datetime.now()))
            self.totalAdded=self.totalAdded+len(entryList)
            print("totalAdded:"+str(self.totalAdded))
            time.sleep(2)
        lock.release()
class SaveList(Thread):
    totalSaved=0
    '''save entry to server'''
    def __init__(self):
        Thread.__init__(self)
    def run(self):
        lock=RLock()
        lock.acquire()
        while True:
            #save list to database,then empty the list
            self.totalSaved=self.totalSaved+len(entryList)
            del entryList[:]
            print("totalSaved:"+str(self.totalSaved))
            time.sleep(3)
        lock.release()

if __name__=="__main__":
    global entryList
    entryList=[]
    addClass= AddToList()
    addClass.start()
    saveClass=SaveList()
    saveClass.start()

result:

totalAdded:2
totalSaved:2
totalAdded:3
totalSaved:3totalAdded:4

totalAdded:6
totalSaved:5
totalAdded:7
totalSaved:6
totalAdded:8
totalAdded:10
totalSaved:8
totalAdded:11
totalSaved:9
totalAdded:12
totalAdded:14
totalSaved:11
totalAdded:15
totalSaved:12
...........
...........
totalAdded:51
totalSaved:39totalAdded:52

totalAdded:54
totalSaved:41
totalAdded:55
totalSaved:42
totalAdded:56
totalAdded:58
totalSaved:44
totalAdded:59
totalSaved:45totalAdded:60
......
......

i anm new to python and searched a lot about threading ,Lock and RLock ,but with no luck. where am wrong?

Great question. Also as a tip to a new python dev, you should avoid using globals at all cost. Also be aware that you're using [class variables instead of instance variables](http://stackoverflow.com/questions/12657867/class-variable-vs-instance-variable-python) which I don't think you meant to. — CornSmith, Jan 03 '14 at 18:08

smeso · Accepted Answer · 2014-01-03T18:53:52.927

To make Lock and RLock work you must use the same object in every thread. The lock objects must have the same "visibility" of the object that you want to "protect".

Here is a new version of you code which should work. It also avoid using things like global variables etc.

import datetime
import time
import threading

class AddToList(threading.Thread):

    def __init__(self, lock, entryList):
        threading.Thread.__init__(self)
        self.totalAdded = 0
        self.entryList = entryList
        self.lock = lock

    def run(self):
        while True:
            self.lock.acquire()
            entryList.append("AddToList at {}".format(datetime.datetime.now()))
            self.totalAdded += 1
            self.lock.release()
            print("totalAdded: {}".format(self.totalAdded))
            time.sleep(2)


class SaveList(threading.Thread):
    def __init__(self, lock, entryList):
        threading.Thread.__init__(self)
        self.totalSaved = 0
        self.entryList = entryList
        self.lock = lock

    def run(self):
        while True:
            self.lock.acquire()
            self.totalSaved += len(self.entryList)
            del self.entryList[:]
            self.lock.release()
            print("totalSaved: {}".format(self.totalSaved))
            time.sleep(3)


if __name__=="__main__":
    lock=threading.Lock()
    entryList=[]

    addClass = AddToList(lock, entryList)
    addClass.start()

    saveClass = SaveList(lock, entryList)
    saveClass.start()

Some things to note:

Use Lock instead of RLock when you don't have any particular needs. RLock is much slower.
As already pointed out by someone it is better avoid using global variables when not needed. Also Class variables should be used only when it makes sense.
When you use a lock you should try to limit as much as possible the code between acquire and release. In you previous code you never release the lock.

thanks for your quick response. i have defined a global lock object and replace the local lock object in each threading, but the result is as same as above. — phiree, Jan 03 '14 at 18:13
Maybe you are assuming that the threads will be executed in a particular order or that those `sleep` will last exactly for the number of seconds that you passed to them. This is not true. — smeso, Jan 03 '14 at 18:20
yes, i don't expect they are equal in every accurate 6 seconds, but they should be equal in some time, and the gap between them won't be get bigger and bigger. — phiree, Jan 03 '14 at 18:26
thanks for your work, but the result is same , the gap get bigger and bigger when running totalAdded: 12 totalAdded: 14 totalSaved: 11 totalAdded: 15 totalSaved: 12 totalAdded: 16 totalAdded: 18 totalSaved: 14 totalAdded: 19 totalSaved: 15 totalAdded: 20 totalAdded: 22 totalSaved: 17: — phiree, Jan 03 '14 at 18:44
`self.totalAdded += len(entryList)` is wrong. You just added only one item, not `len(entryList)` ones. — Armin Rigo, Jan 03 '14 at 18:47

multi threads modify a global list in python

1 Answers1