I have a list of objects and I want to call a method of these objects iteratively in a for loop but to speed up things I need to run this on parallel. Here is an example:
class myObj:
def __init__(self):
self.x = 0
def go(self):
self.x += 1
myList = [myObj() for i in range(100)]
for t in range(1000):
for obj_ in myList:
obj_.go()
How can I run the second loop in parallel? because these objects are distinct it should be possible.
One way to do this using python 3.x would be to use the multiprocessing.Pool and map method to apply the myObj.go method to every element in your myList. The processes parameter for constructing Pool can be changed according to the number of parallel processes you would like to use, normally based on the available number of cores on your computer. Note that in order to keep the result of updating myObj.x on each iteration, I have revised the go method to return self:
import multiprocessing as mp
class myObj:
def __init__(self):
self.x = 0
def go(self):
self.x += 1
return self
if __name__ == "__main__":
myList = [myObj() for i in range(100)]
with mp.Pool(processes=4) as pool:
for t in range(1000):
myList = pool.map(myObj.go, myList)
You could also have a read of this answer for some more possible approaches.
