Python iterating over a list in parallel?

Question

I have a list created with no more than 8 items in it, right now I loop over the item with a standard for loop "for item in list:" I am wondering if there is a way to check all of the elements at the exact same time? I have seen some posts about using zip to iterate over 2 lists at the same time which could help.

If there is no way to iterate over a lists elements at the same time I would guess I need to split one single list into 4 separate lists and iterate 2 at the same time using zip and do the other 2 on a separate thread, that seems like it could be a clunky fix though.

Any help would be appreciated.

Edit Sorry to be vague, I have a list

apples = ['green', 'green', 'green', 'red']

right now I use a for loop to go over each element,

for apple in apples:
    checkAppleColour(apple)
    #Do some more stuff depending on colour....

right now it loops over each element in apple one by one, what I would like to be able to do is check each of the items at the exact same time for a colour check as every millisecond counts (example code used, but principle is exactly the same for what I am trying to achieve).

Performance is key here I don't mind if its threading I have to use, but I need maximum speed / efficiency for checking each element and my guess is parallelism is the way to do it, I am just not quite sure how.

what do you mean exactly by `check all of the elements at the exact same time` ? What do you actually want to do ? — Overdrivr, Feb 10 '16 at 15:21
Can you add some example code to clarify exactly what you want to do. — Håken Lid, Feb 10 '16 at 15:23
I'll second @Overdrivr what are you trying to acheive/do? are you looking for multiprocessing solution? are you trying to compare several elements to one another? — Fanchi, Feb 10 '16 at 15:23
`zip` is only used when you need data from two lists in the same `for` loop, it doesn't do any parallel processing. Start a thread for each element of your list if you want them processed independently and in parallel — maahl, Feb 10 '16 at 15:31
`my guess is parallelism is the way to do it` do not guess, profile and measure it. — Łukasz Rogalski, Feb 10 '16 at 17:04

score 7 · Answer 1 · answered Feb 10 '16 at 15:31

If you can map a function onto the list, parallelism is easy.

def addOne (x):
    return x + 1

The default map does things on one thread.

map(addOne, range(1, 4)) #When iterated over produces [2, 3, 4]

If your problem is a map, it means that it's trivially parallel.

from multiprocessing import Pool

pool = Pool()
pool.map(addOne, range(1, 4)) ##When iterated over produces [2, 3, 4]

This uses multiple processes. multiprocessing.dummy.Pool is a thread pool. See this article for a great introduction to the rationale behind both.

Python iterating over a list in parallel?

1 Answers1

Linked