Python threading: how to use return values of external scripts?

Question

(I found a decent solution here for this, but unfortunately I'm using IronPython which does not implement the mutliprocessing module ...)

Driving script Threader.py will call Worker.py's single function twice, using the threading module. Its single function just fetches a dictionary of data.

Roughly speaking:

Worker.py

def GetDict():
    :
    :
    :
    return theDict

Threader.py

import threading
from Worker import GetDict
    :
    :
    :
def ThreadStart():
    t = threading.Thread(target=GetDict)
    t.start()
    :
    :

In the driver script Threader.py, I want to be able to operate on the two dictionaries outputted by the 2 instances of Worker.py.

The accepted answer here involving the Queue module seems to be what I need in terms of accessing return values, but this is written from the point of view of everthing being doen in a single script. How do I go about making the return values of the function called in Worker.py available to Threader.py (or any other script for that matter)?

Many thanks

Do you mean that you are launching `Worker.py` using `subprocess` or `os.system`? — ubik, May 14 '12 at 09:48
I've reworded my question, thanks - I'm not actually launching Worker.py, moreover its only function GetDict() is being called by Threader.py — Pyderman, May 14 '12 at 10:01

mata · Accepted Answer · 2012-05-14T11:08:29.327

another way to do what you want (without using a Queue) would be by using the concurrent.futures module (from python3.2, for earlier versions there is a backport).

using this, your example would work like this:

from concurrent import futures

def GetDict():
    return {'foo':'bar'}

# imports ...
# from Worker import GetDict

def ThreadStart():
    executor = futures.ThreadPoolExecutor(max_workers=4)
    future = executor.submit(GetDict)
    print(future.result()) # blocks until GetDict finished

    # or doing more then one:
    jobs = [executor.submit(GetDict) for i in range(10)]
    for j in jobs:
        print(j.result())

if __name__ == '__main__':
    ThreadStart()

edit:

something similar woule be to use your own thread to execute the target function and save it's return value, something like this:

from threading import Thread

def GetDict():
    return {'foo':'bar'}

# imports ...
# from Worker import GetDict

class WorkerThread(Thread):

    def __init__(self, fnc, *args, **kwargs):
        super(WorkerThread, self).__init__()
        self.fnc = fnc
        self.args = args
        self.kwargs = kwargs

    def run(self):
        self.result = self.fnc(*self.args, **self.kwargs)


def ThreadStart():
    jobs = [WorkerThread(GetDict) for i in range(10)]
    for j in jobs:
        j.start()
    for j in jobs:
        j.join()
        print(j.result)

if __name__ == '__main__':
    ThreadStart()

Thanks, I'm using IronPython though, so neither this nor the backport are options for me. — Pyderman, May 14 '12 at 10:11
yes, you're right, i've tried it with IronPython now. but also by removing the line `from concurrent.futures.process import ProcessPoolExecutor` in `concurrent.futures.__init__.py` it starts working, just without multiprocessing support. would be great if they had added a try/catch here... — mata, May 14 '12 at 10:46

Python threading: how to use return values of external scripts?

1 Answers1