How do I modify a list inside a thread?

Question

I need to modify a list by adding some elements at the end from a thread.

This is my code:

def go():
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]  
      alist.append(ip)  #<================== her i use the liste

if __name__ == '__main__':      
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname) 
    text = text[end:len(text)]  

  for i in range(10):   
    threading.Thread(target=go,args=(alist)).start() #<====== i give the list as argument

  print alist

The last print statement display an empty list, []. Any ideas?

You need to have some synchronization, probably with mutexes. — Basile Starynkevitch, Mar 05 '14 at 20:28
can i have an exemples of using muteness, i'm sorry but i am new with python — user3383192, Mar 05 '14 at 20:42
Place alist = [] outside main,above your go() function for instance, and then use it in main. I would recommend that you do the same with instantiating queue;queueCommand = Queue.Queue(). It might be easier if you use this within a threading class and define a separate function for reading the file. Here's a link to an SO answer that might be useful http://stackoverflow.com/questions/6286235/multiple-threads-in-python — user1749431, Mar 05 '14 at 21:02

score 2 · Accepted Answer · answered Mar 05 '14 at 20:54

You have a few problems.

You specify alist as the args, but you need to pass it as a tuple which it looks like you tried to do, but a one item tuple would look like this (alist,). Right now you are just using the alist global which is likely not what you want.
Your go method does not expect an argument (namely, the alist).
To be thread safe, I believe you will need to use some sort of semaphore/mutex/lock primitive. The threading module comes with a Lock implementation which you can use to restrict access to alist during the append operation.
Most importantly, you are not waiting for your threads to finish before you print the result. To wait for thread to finish you need to call .join() on the thread.

I would probably opt for using another Queue instance to put the results into, then you could read all from the queue to construct your list once the threads are complete.

Here is an updated version (working) of your code. Like I said, I would probably opt to use a Queue instead, and I have not used the threading module much since I switched to eventlet/gevent... so there may be ways to improve on what I provided.

import threading
import Queue
import subprocess

lock = threading.Lock()

def go(alist):
  while queueCommand.qsize() > 0:
      command = queueCommand.get(False)
      res = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=None, shell=True)
      output = res.communicate()
      output = str(output)
      star = output.find("address") + 8
      en = output.find(",")
      ip = output[star:en-3]
      lock.acquire()
      alist.append(ip)  #<================== her i use the liste
      lock.release()

def foo(alist):
    alist.append("bar")

if __name__ == '__main__':
  with open ("icq.txt","r") as myfile:
    text = myfile.read()
  end = 0
  alist = []
  queueCommand = Queue.Queue()
  while True:
    start = text.find("href=") + 13
    if start == 12:
      break
    end = text.find("/",start)
    if text[start:end].find("icq.com") != -1:
      hostname="host "+ text[start:end]
      queueCommand.put(hostname)
    text = text[end:len(text)]

  threads = []
  for i in range(10):
    thread = threading.Thread(target=go,args=(alist,)) #<====== i give the list as argument)
    thread.start()
    threads.append(thread)
  for thread in threads:
    thread.join()

  print alist

How do I modify a list inside a thread?

1 Answers1