Loop through multiple lists in specific intervals

Question

I have two lists. One with names, and one with numbers that correspond with a name in the first list (corresponding name and number are at the same index point in each list). I need to reference each name and number in a url that can only handle 25 different names & points at a time.

pointNames = ['name1', 'name2', 'name3']
points = ['1', '2', '3']  #yes, the numbers are meant to be strings

My actual lists have roughly 600 values in each. What I'm trying to do is loop through each list at the same time, but in increments of 25. I'm able to do this with a single list using the following:

def chunker(seq, size):
    return (seq[pos:pos + size] for pos in range(0, len(seq), size))

for group in chunker(pointNames, 25):
    print (group)

This prints multiple groups of 25 values from the list until it has gone through the entire list. I want to do exactly this, but with two lists. I'm able to print each list entirely with for(point, name) in zip(points, pointNames): but it I need it in groups of 25.

I've also tried combining the two lists into a dictionary:

dictionary = dict(zip(points, pointNames))

for group in chunker(dictionary, 25):
    print (group)

but i get the following error:

TypeError: unhashable type: 'slice'

score 2 · Answer 1 · answered Mar 27 '18 at 18:23

How about this relatively minimal change to your first function:

def chunker(seq1, seq2, size):
    seq = list(zip(seq1, seq2))
    return (seq[pos:pos + size] for pos in range(0, len(seq), size))

Call as follows:

for group in chunker(pointNames, points, 25):
    print(group)

Jacques Gaudin · Accepted Answer · 2018-03-27T18:54:17.090

2

A generator would be more efficient:

import itertools

def chunker(size, *seq):
    seq = zip(*seq)
    while True:
        val = list(itertools.islice(seq, size))
        if not val:
            break
        yield val


for group in chunker(2, pointNames, points):
    print(group)

gen_groups = chunker(2, pointNames, points, pointNames, points)
group = next(gen_groups)
print(group)

Using *seq allows you to give any number of list as parameters.

edited Mar 27 '18 at 18:54

answered Mar 27 '18 at 18:35

Jacques Gaudin

15,779
10
54
75

This works great. Allows me to reference an individual name or number from a given set of 25 in each loop with `group[n][n]`. One correction I made was on your `gen_groups` variable - removed the duplicate `pointNames, points` – Aaron Mar 27 '18 at 18:55
@Aaron That was just to show you can pass as many lists as you want. – Jacques Gaudin Mar 27 '18 at 18:56
Ah ok, I understand. Thank you – Aaron Mar 27 '18 at 18:59

score 2 · Answer 3 · answered Mar 27 '18 at 18:56

Itertools can slice an iterator (or generator) into chunks, together with a small helper function to keep going until it is done:

import itertools

# helper function, https://stackoverflow.com/a/24527424
def chunks(iterable, size=10):
    iterator = iter(iterable)
    for first in iterator:
        yield itertools.chain([first], itertools.islice(iterator, size - 1))


# 600 points and pointNames
points = (str(i) for i in range(600))
pointNames = ('name ' + str(i) for i in range(600))

# work with the chunks
for chunk in chunks(zip(pointNames, points), 25):
    print('-' * 40)
    print(list(chunk))

Loop through multiple lists in specific intervals

3 Answers3