Can python threads access variables in the namespace?

Question

I have a script that creates a bunch of threads, runs a program to use the threads to run tasks from a queue, and returns something from each thread. I want to count how many of these returned successfully, so I set a variable "successful=0" and increment it every time the queue reports a task completed successfully.

However, I'm getting "UnboundLocalError: local variable 'successful' referenced before assignment"

What's going on?

Here's some example code:

successful = 0

q = Queue(200)
for i in range(100):
    t=Thread(target=foo)
    t.daemon=True
    t.start()
def foo():
    while True:
        task=q.get()
        #do some work
        print task
        successful+=1 # triggers an error
        q.task_done()
for i in range(100):
    q.put("Foo")
q.join()
print successful

Answer 1

successful+=1

is not a thread-safe operation. With multiple threads trying to increment a shared global variable, collisions may happen and successful will not be incremented properly.

To avoid this error, use a lock:

lock = threading.Lock()
def foo():
    global successful
    while True:
        ...
        with lock:
            successful+=1 

---

Here is some code to demonstrate that x += 1 is not threadsafe:

import threading
lock = threading.Lock()
x = 0
def foo():
   global x
   for i in xrange(1000000):
       # with lock:    # Uncomment this to get the right answer
            x += 1
threads = [threading.Thread(target=foo), threading.Thread(target=foo)]
for t in threads:
    t.daemon = True    
    t.start()
for t in threads:
    t.join()

print(x)

yields:

% test.py 
1539065
% test.py 
1436487

These results do not agree and are less than the expected 2000000. Uncommenting the lock yields the correct result.

Answer 2

The problem happens because a variable that is assigned inside a function is considered to be local to that function. If you want to modify the value of a variable successfull, that is created outside a foo, you need to explicitly inform the interpreter that you are going to work with a global variable inside a function. This can be done in the following way:

def foo():
    global successfull
    while True:
        task=q.get()
        #do some work
        print task
        successful+=1 # triggers an error
        q.task_done()

Now the code should work as expected.

Answer 3

Based on Python variable scope error,

I should have put "global successful" under "def foo():".

Oops.