Distinguishing the return values from threads when a queue is used

Question

This is my code :

  queue=Queue.Queue()
    isbn=str(9789382711056)
    thread1= Thread(target = amazon, args=[isbn,queue])
    thread2= Thread(target = bookadda, args=[isbn,queue])
    thread3= Thread(target = infibeam, args=[isbn,queue])
    def jo():
        thread1.join()
        thread2.join()
        thread3.join()
    thread1.start()
    thread2.start()
    thread3.start()

   jo()

And each of those functions put a list in the queue ( queue.put(list) ). Code for one of the function:

def infibeam(isbn,queue):
    #b_price and b_avail are obtained from some where else
    blist =[b_price,b_avail]
    queue.put(blist)

The other methods are also similar to this.

I get all the lists in the queue , but how can I know which method has returned which list ? Please help me. This might be a very silly question, but I am new to python. Thanks in Advance.

score 3 · Accepted Answer · edited May 23 '17 at 10:25

If you use a multiprocessing.ThreadPool, then you could use pool.map:

import multiprocessing.pool as mpool

def worker(target, isbn):
    return target(isbn)

def amazon(isbn):
    ...

def bookadda(isbn):
    ...

def infibeam(isbn):
    #b_price and b_avail are obtained from some where else
    return [b_price, b_avail]


pool = mpool.ThreadPool()
isbn = str(9789382711056)
args = [(target, isbn) for target in (amazon,bookadda,infibeam)]
result = pool.map(worker, args)

The multiprocessing.ThreadPool has the same API as multiprocessing.Pool, except that the pool is composed of threads instead of processes.

The order of the items in result correspond to the order of the items in args.

In Python3 you could use a concurrent.futures.ThreadPoolExecutor:

import concurrent.futures as CF

with CF.ThreadPoolExecutor() as executor:
    future = executor.map(worker, args)
    result = list(future)

There is also an example using ThreadPoolExecutor in the documentation showing how to associate the argument with the result.

score 1 · Answer 2 · answered Dec 28 '13 at 13:41

1

Have each thread label its return value with a unique tag (a string, the thread function itself, etc). Then have the main thread examine the tags to figure out which function has produced which list.

As a variant of the above, have the thread functions place their return values into a dictionary (again, keyed by a unique tag of some sort).

answered Dec 28 '13 at 13:41

NPE

486,780
108
951
1,012

This is do-able , but I have around 20 such functions and It would be difficult to search each list to check where the list belongs to. Is there any alternative other than using queue ? – GingerNinja23 Dec 28 '13 at 13:48
@GingerNinja23: Encapsulate all this functionality in a class and use it in the 20 places. – NPE Dec 28 '13 at 13:53

Distinguishing the return values from threads when a queue is used

2 Answers2