Parallelize this list comprehension in Python

Question

I'm trying to make this statement run in parallel (on 4 threads).

[x for x in obj_list if x.attribute == given_attribute]

Any help would be appreciated.

I found this question useful for other type of comprehension, but not for filtering like in this case.

See the docs [here](https://docs.python.org/3/library/concurrent.futures.html#processpoolexecutor-example), you can replace `ProcessPoolExecutor` with `ThreadPoolExecutor`. — Adam, Aug 17 '15 at 16:21

score 3 · Accepted Answer · answered Aug 17 '15 at 16:41

You can use a Pool as described in the example you provided. This sort of works, but you have to remove the None result afterwards:

import multiprocessing as mp

class Thing:
    def __init__(self, y):
        self.attribute = y

def square(thing, given_attribute):
    if thing.attribute == given_attribute:
        return thing

given_attribute = 4
x = [Thing(i) for i in range(10)]  # List of objects to process

if __name__ == '__main__':
    pool = mp.Pool(processes=4)
    results = [pool.apply(square, args=(x[i], given_attribute, )) for i in range(10)]
    r = [i for i in results if i is not None]  # Remove the None results
    print r

Parallelize this list comprehension in Python

1 Answers1