Python Multithreading Not Working

Question

I am new to learning python and got an exercise to create a multithreaded script to take a list of 10 public ftp servers and connect to them anonymously and just do a directory listing. I have the following code and it works when i use the ftp connect within the run function but when i try to create an "ftp" function and utilize it keeps erroring out and then the terminal gets stuck and can't kill the program or get out, which i can't figure out why that keeps happening either?

!/usr/bin/python

import threading
import Queue
import time
from ftplib import FTP



sites = ["speedtest.tele2.net", "test.rebex.net", "test.talia.net", "ftp.swfwmd.state.fl.us", "ftp.heanet.ie", "ftp.rediris.es", "ftp.ch.freebsd.org",  "ftp.mirror.nl", "ftp.ussg.iu.edu", "ftp.uni-bayreu$


class WorkerThread(threading.Thread) :

        def __init__(self, queue) :
                threading.Thread.__init__(self)
                self.queue = queue

        #def ftp(ip) :
        #       server = FTP(ip)
        #       server.login()
        #       server.retrlines('LIST')

        def run(self) :
                print "In WorkerThread"
                while True :
                        counter = self.queue.get()
                        print "Connecting to FTP Server %s" % counter
                        #self.ftp(counter)
                        #print "Ordered to sleep for %d seconds!" % counter
                        #time.sleep(counter)
                        #print "Finished sleeping for %d seconds" % counter

                        server = FTP(counter)
                        server.login()
                        server.retrlines('LIST')
                        self.queue.task_done()

queue = Queue.Queue()

for i in range(10) :
        print "Creating WorkerThread : %d" % i
        worker = WorkerThread(queue)
        worker.setDaemon(True)
        worker.start()
        print "WorkerThread %d Created!" % i


for j in sites :
        queue.put(j)


queue.join()

print "All Tasks Over!"

You should put the exception traceback to get something more on the issue, either way at first glance it looks to me your _ftp_ method doesn't get the right arguments. As it is written you-ll get the class instance, which is always passed as first arg, into the ip argument. I think you should declare the method as `def ftp(self, ip)`. — steppo, Jun 08 '17 at 19:49
your right that worked, when i defined it as def ftp(self, ip) but for some reason when it finishes, it just gets stuck and i can't even ctrl-c out of it — Gerald Germain, Jun 08 '17 at 19:52

score 0 · Answer 1 · answered Jun 08 '17 at 20:26

As suggested by:

Is there any way to kill a Thread in Python?

you should put a stop condition a make the threads check on it. Together with the join it allows for the thread to be terminated gracefully. Without entering into some other implication try the code below.

#!/usr/bin/python

import threading
import Queue
import time
from ftplib import FTP


sites = ["speedtest.tele2.net"]


class WorkerThread(threading.Thread):

        def __init__(self, queue):
                threading.Thread.__init__(self)
                self.queue = queue
                self._stop = threading.Event()

        def ftp(self, ip):
            server = FTP(ip)
            server.login()
            server.retrlines('LIST')

        def run(self):
                print "In WorkerThread"
                while not self.stopped():
                    counter = self.queue.get()
                    print "Connecting to FTP Server %s" % counter
                    self.ftp(counter)
                    self.queue.task_done()

        def stop(self):
            self._stop.set()

        def stopped(self):
            return self._stop.is_set()


if __name__ == '__main__':

    queue = Queue.Queue()

    for i in range(10):
            print "Creating WorkerThread : %d" % i
            worker = WorkerThread(queue)
            worker.setDaemon(True)
            worker.start()
            worker.stop()
            print "WorkerThread %d Created!" % i

    for j in sites:
            queue.put(j)

    queue.join()

    print "All Tasks Over!"

wow, thanks for the code, i tried it, but i still can't get out of it, when i hit ctrl-c, is that something normal? — Gerald Germain, Jun 09 '17 at 13:09
It is quite normal in that your process doesn't handle outer signals. Nonetheless I expected the process to terminate after the list had been consumed. Checked that out on a single server, but I must admit this is a tipical issue when dealing with thread: debugging becomes pretty complicated. — steppo, Jun 09 '17 at 15:56

Python Multithreading Not Working

1 Answers1