Using multiple variables over a single list in python (for loop)

Question

I am working on a python script, which requires reading values from a list in for loop and performing some operations on it. currently it looks like this:

for i in someList:
    # do some operation here

Now, this particular operation takes hours to complete, so what i want to be able to do is use multi-threading/sub-processes and process multiple items in parallel. So is there a way that I can run the for loop like:

for i,j,k in someList:
    #do some operation here

i know this syntax can be used to iterate over multiple lists simultaneously, but I want to be able to use multiple items of the same list. What i want from the loop is that the value of i,j and k update in the following manner:

i = someList[1], j = someList[2], k = someList[3]

in the next iteration:

i = someList[4], j = someList[5], k = someList[6]

you can only do `for i,j,k in someList:` if `someList` is a list with list. — MEdwin, Jun 19 '19 at 10:22
unless do something operation is not clear, it doesn't matter if you do by this way or process by single element. — sahasrara62, Jun 19 '19 at 10:23
@prashantrana do something is a function call, which i want to run in parallel, using multiprocessing. i dont see how thats relevant to the question though. you can consider it a print(i), print(j), print(k) operation for that matter. — Khizar Amin, Jun 19 '19 at 10:29
@KhizarAmin, you don't need to split it to groups to use multiprocessing — Olvin Roght, Jun 19 '19 at 10:30
@OlvinRoght what i am trying to achieve is something like: p1 = multiprocessing.Process(target=my_Func, args=(i, )) p2 = multiprocessing.Process(target=my_Func, args=(j, )) p3 = multiprocessing.Process(target=my_Func, args=(k, )) and so on. Without multiprocessing this function is taking about an hour per iteration, so over the complete list, it takes nearly a whole day, which is why i am trying to convert it into a multiprocessing task. — Khizar Amin, Jun 19 '19 at 10:34
@KhizarAmin, you can do like that `for i, j, k in zip(some_list[::3], some_list[1::3], some_list[2::3]):` but it seems like it useless. — Olvin Roght, Jun 19 '19 at 10:37
@KhizarAmin, or like this `for i, j, k in (some_list[i:i + 3] for i in range(0, len(some_list), 3)):` — Olvin Roght, Jun 19 '19 at 10:47
TypeCasting is very **important** on any programing language ! You actually mixed `dreams` and `real life`. if you want use pure python, dont fill any values with `None`. You can't run any iteration over `UNDEFINED`. Required selfcheck in any iteration. Some list never started with zero elements(`a_list = [range(3),range(3)....]`) — dsgdfg, Jun 19 '19 at 10:47
You could e.g. `zip` an `iter` of the list with itself and loop over that: `for a, b, c in zip(*[iter(lst)]*3):` — tobias_k, Jun 19 '19 at 12:08

lprakashv · Accepted Answer · 2019-06-19T12:24:51.813

We can write our custom iterating n_list class which can be used for hopping (default hop = 1, like normal lists) iterations:

class n_list:
    def __init__(self, lst, n=1):
        self.n = n
        self.lst = lst
        self.index = 0

    def __iter__(self):
        return self

    def __next__(self):
        l = self.lst[self.index:self.index+self.n:]
        if len(l) == 0:
            raise StopIteration
        self.index += self.n
        # 5 * [None] -> [None,None,None,None,None]
        # ([1,2,3] + 5 * [None]) -> [1,2,3,None,None,None,None,None]
        # ([1,2,3] + 5 * [None])[:5] -> [1,2,3,None,None]
        # tuple([1,2,3,None,None]) -> (1,2,3,None,None)
        return tuple((l + self.n * [None])[:self.n])

Testing our n_list with a tuple size of 3:

test_list = [10, 20, 30, 40, 50, 60, 70, 80]
for a, b, c in n_list(test_list, 3):
    print('a: ' + str(a) + ', b: ' + str(b) + ', c: ' + str(c))

Output:

a: 10, b: 20, c: 30
a: 40, b: 50, c: 60
a: 70, b: 80, c: None

Using multiple variables over a single list in python (for loop)

1 Answers1