Unclear behavior of multithreading application: can't exit thread

Question

Multithreading app on python 2.7. I use "threading" and "thread" libraries.

Main thread had started other 10 threads, which do some work. They have one shared class with data (singletone). I don't use any thread blocking, and it's seems good. Main thread starting this threads with "start" and "join" methods of threading class. One of ten threads, was starting every 10 seconds and do some math calculation. When the work is complete, the thread invoke "thread.exit()".

And sometimes main thread did not have the result of the one thread. The thread is end! And all strings of code are complete, but main thread stops on "join" instruction and did not response.

p.s. I'm not native english speacker, and discribe that problem was very difficult. Please be tolerant.

Code example:

while True: 

all_result = check_is_all_results()
time.sleep(1)

if (all_result):
    print app_data.task_table
    app_data.flag_of_close = True
    time.sleep(2)  # Задержка на всякий случай

if (app_data.flag_of_close):
    terminate()
print u"TEST"

if len(app_data.ip_table[app_data.cfg.MY_IP]['tasks']):  
    if (app_data.cfg.MULTITHREADING or app_data.complete_task.is_set()):  
        job = Worker(app_data, SRV.taskResultSendToSlaves, app_data.ip_table[app_data.cfg.MY_IP]['tasks'].pop())
        job.setDaemon(True)
        job.start()


###########################################################
class Worker(threading.Thread):

def __init__(self, data, sender, taskname):
    self.data = data
    self.sender = sender
    self.taskname = taskname
    threading.Thread.__init__(self)

def run(self):
    import thread
    self.data.complete_task.clear()
    tick_before = time.time()
    startupinfo = subprocess.STARTUPINFO()
    startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
    startupinfo.wShowWindow = subprocess.SW_HIDE
    p = subprocess.Popen(self.data.cfg.PATH_INTERPRETER + " " + self.data.cfg.PATH_TASKS + self.taskname, startupinfo=startupinfo, shell=False, stdout=subprocess.PIPE)
    job_result, err = p.communicate()
    tick_after = time.time()
    work_time = tick_after - tick_before
    self.data.task_table[self.taskname]['status'] = 'complete'
    self.data.task_table[self.taskname]['result'] = job_result
    self.data.task_table[self.taskname]['time'] = work_time
    tr = threading.Thread(target=self.sender, name="SENDER", args=(self.taskname, ))
    tr.setDaemon(True)
    tr.start()
    tr.join()
    self.data.complete_task.set()
    thread.exit()

Sometimes main infinite loop, which calls Worker, does not print "TEST", and does not response.

Not sure how you are returning data from the thread, but you can try something like this: http://stackoverflow.com/a/6894023/276949 — Martin Konecny, Aug 16 '15 at 16:08

score 0 · Answer 1 · edited May 23 '17 at 11:44

Your worker threads are spawning subprocesses. Unfortunately, this never works right, because this is first done with a fork that only copies the executing thread from the parent process. Sorry, but your program will not be reliable until you restructure it. Here is some background information with links to more information:

https://stackoverflow.com/a/32107436/3577601

Status of mixing multiprocessing and threading in Python

https://stackoverflow.com/a/6079669/3577601

Unclear behavior of multithreading application: can't exit thread

1 Answers1