What's the simplest way to implement fire-and-forget parallel threads in Python?

Question

I have a program with a tkinter GUI (Python 2.7) from which the user can launch some more or less calculation-intense tasks. Most of those result either in a file written to disk or an interactive pyplot window showing results, but none of them feed back into the main task.

I've never done multithreading before and am trying to decide which library to go for. subprocess seems to be for system calls (which this is not), multiprocessing seems to be concerned with parallel execution of larger tasks (with pools and queues and such), and threading ... I've looked through the official documentation but am somewhat unclear how I would use this.

The ideal solution for me would be something that would "simply" phrase the function call which triggers calculating and plotting some data in a way that it will be executed independent of the main program so the user can keep doing their thing without waiting for the function to finish (which usually takes a few seconds up to a minute) -- the plot would just eventually pop up.

Update Turns out the task I wanted to launch in parallel contains bound methods, thus is not picklable and can't be used in parallel. I need to deal with some other business before I'll have time to figure out how to change that -- I'll get back to this, though!

score 1 · Answer 1 · answered Jul 07 '15 at 10:23

1

The multiprocessing library is probably your best bet, just create a process and start it. From the manual: https://docs.python.org/2/library/multiprocessing.html

from multiprocessing import Process

def f(name):
    print 'hello', name

if __name__ == '__main__':
    p = Process(target=f, args=('bob',))
    p.start()

On exit you probably do want to wait for the results, so add the join() method:

p.join()

Or if you simply want to wait for a short while:

p.join(timeout=1)

answered Jul 07 '15 at 10:23

Wolph

78,177
11
137
148

I'm getting an error message about Pickle not working on one of the objects involved in the task ... I suppose ``multiprocessing`` is using Pickle to create a "copy" of the objects involved in the subprocess? – Zak Jul 08 '15 at 14:05
@Zak: essentially, yes. The multiprocessing library executes everything in a separate process and it passes everything along as pickled objects. If that's a problem you can use threads instead but in that case everything will be executed using the same CPU core so it can slow down your main process. – Wolph Jul 08 '15 at 14:55
About `pickle` in `multiprocessing`: https://stackoverflow.com/q/27318290 – djvg Jan 18 '22 at 14:11

score 0 · Answer 2 · answered Jul 07 '15 at 10:29

Wondering if this would help. This is a simple pattern I have used to execute threads in Python

from threading import Thread

class Operation(Thread):
    def __init__(self):
        """Initialize"""
    def run(self):
        """Implement the run method. This will get executed when you 
           instantiate and call the start method"""

def main():
    """ main program"""
    mythread = Operation()
    mythread.start()
    ...
    mythread.join()

What's the simplest way to implement fire-and-forget parallel threads in Python?

2 Answers2