79

Let's say I have a list of arbitrary length, L:

L = list(range(1000))

What is the best way to split that list into groups of n? This is the best structure that I have been able to come up with, and for some reason it does not feel like it is the best way of accomplishing the task:

n = 25
for i in range(0, len(L), n):
    chunk = L[i:i+25]

Is there a built-in to do this I'm missing?

Edit: Early answers are reworking my for loop into a listcomp, which is not the idea; you're basically giving me my exact answer back in a different form. I'm seeing if there's an alternate means to accomplish this, like a hypothetical .split on lists or something. I also do use this as a generator in some code that I wrote last night:

def split_list(L, n):
    assert type(L) is list, "L is not a list"
    for i in range(0, len(L), n):
        yield L[i:i+n]
Jed Smith
  • 15,584
  • 8
  • 52
  • 59
  • 1
    create a generator for more Pythonic. But us for me - it is normal code :) – Oduvan Oct 26 '09 at 14:04
  • You can create your own class for storing a list and code it there, then you'd have a more direct approach... but I doubt it would change anything. Lists don't have a .split() function, at least not in python 2.x ;) – kender Oct 26 '09 at 14:14
  • @Jurily Pythonic idioms -- http://stackoverflow.com/questions/58968/what-defines-pythonian-or-pythonic – Jed Smith Oct 26 '09 at 14:16
  • 1
    @Jurily check out http://stackoverflow.com/questions/58968/what-defines-pythonian-or-pythonic – Steg Oct 26 '09 at 14:17
  • 5
    this question makes me think we need `itertools.split(iterable, itervallen)` – u0b34a0f6ae Oct 26 '09 at 14:19
  • @Jed: could you explain why the list comprehensions-equivalent of your original code doesn't seem pythonic to you? – SilentGhost Oct 26 '09 at 14:29
  • @SilentGhost I will elaborate in the answer – Jed Smith Oct 26 '09 at 14:32
  • i find it a little disturbing how both list comprehension based answers were downvoted – Steg Oct 26 '09 at 14:58
  • 1
    @Jed: that's not an answer. No one argues that your code is not pythonic, however, the list-comprehensions version is simple, idiomatic and is fairly extensible. While other tricks posted work and might be even more efficient, I don't see how any of them are pythonic. P.S. please don't edit your answer if you're replying to a comment, it makes conversation very hard to follow. – SilentGhost Oct 26 '09 at 15:08
  • @SilentGhost: Well, this is an answer: *Is there a built-in I'm missing?* – Jed Smith Oct 26 '09 at 16:29
  • I've reworked the question to appease you, and removed "Pythonic" from the title and summary. Better? – Jed Smith Oct 26 '09 at 16:32
  • 2
    @SilentGhost, I agree that those tricks may not seem pythonic and are hard to understand for beginners. But they are suggested in python documentation as a recipe. That counts for something. – Nadia Alramli Oct 26 '09 at 16:38
  • 1
    @Nadia: 100% agreed. The trouble is "Pythonic" is open to interpretation, and some people feel that their interpretation is the only way. – Jed Smith Oct 26 '09 at 16:45
  • why not use i+n if you assigned n = 25 – David Andrei Ned Mar 29 '17 at 10:06

6 Answers6

160

Here you go:

list_of_groups = zip(*(iter(the_list),) * group_size)

Example:

print zip(*(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]

If the number of elements is not divisible by N but you still want to include them you can use izip_longest but it is only available since python 2.6

izip_longest(*(iter(range(10)),) * 3)

The result is a generator so you need to convert it into a list if you want to print it.

Finally, if you don't have python 2.6 and stuck with an older version but you still want to have the same result you can use map:

print map(None, *(iter(range(10)),) * 3)
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

I'd like to add some speed comparison between the different methods presented so far:

python -m timeit -s 'from itertools import izip_longest; L = range(1000)' 'list(izip_longest(*(iter(L),) * 3))'
10000 loops, best of 3: 47.1 usec per loop

python -m timeit -s 'L = range(1000)' 'zip(*(iter(L),) * 3)'
10000 loops, best of 3: 50.1 usec per loop

python -m timeit -s 'L = range(1000)' 'map(None, *(iter(L),) * 3)'
10000 loops, best of 3: 50.7 usec per loop

python -m timeit -s 'L = range(1000)' '[L[i:i+3] for i in range(0, len(L), 3)]'
10000 loops, best of 3: 157 usec per loop

python -m timeit -s 'import itertools; L = range(1000)' '[list(group) for key, group in itertools.groupby(L, lambda k: k//3)]'
1000 loops, best of 3: 1.41 msec per loop

The list comprehension and the group by methods are clearly slower than zip, izip_longest and map

Nadia Alramli
  • 111,714
  • 37
  • 173
  • 152
  • 22
    Your Python skills scare me a bit. :) Good answer! – Jed Smith Oct 26 '09 at 14:18
  • 4
    This should be the accepted answer. I couldn't remember the map and zip idioms for this, though I recalled they were the 'correct' way and this came up in a websearch -- thanks. – Jonathan Vanasco Apr 25 '12 at 04:31
  • 2
    +1 for benchmarks on all the solutions. That's something people miss a lot in Python answers, and as one sees, it makes a very very big difference which method you use. – yo' Feb 14 '14 at 13:16
  • 3
    This works by passing multiple references to the same iterator to zip. It's a crazy hack. – Zags Mar 18 '14 at 22:19
  • 7
    A bit of explaination: `(iter(the_list),)` is a tuple. `(iter(the_list),)*3` is a tuple with three references to the same iterator. We use a starred expression `*(...)` to pass the three references to the iterators to `zip` as arguments. `zip` basically does a matrix transpose of the arguments given. – Tomas G. Aug 18 '20 at 07:51
74

How about:

>>> n = 2
>>> l = [1,2,3,4,5,6,7,8,9]
>>> [l[i:i+n] for i in range(0, len(l), n)]
[[1, 2], [3, 4], [5, 6], [7, 8], [9]]
Nicolas Gervais
  • 33,817
  • 13
  • 115
  • 143
Steg
  • 10,000
  • 3
  • 24
  • 17
56

A Python recipe (In Python 2.6, use itertools.izip_longest):

def grouper(n, iterable, fillvalue=None):
    "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return itertools.zip_longest(*args, fillvalue=fillvalue)

Example usage:

>>> list(grouper(3, range(9)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8)]
>>> list(grouper(3, range(10)))
[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

If you want the last group to be shorter than the others instead of padded with fillvalue, then you could e.g. change the code like this:

>>> def mygrouper(n, iterable):
...     args = [iter(iterable)] * n
...     return ([e for e in t if e != None] for t in itertools.zip_longest(*args))
... 
>>> list(mygrouper(3, range(9)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
>>> list(mygrouper(3, range(10)))
[[0, 1, 2], [3, 4, 5], [6, 7, 8], [9]]
Stephan202
  • 59,965
  • 13
  • 127
  • 133
  • 1
    Accepted because you got to it first. I'm not sure which is more Pythonic, as readability is important as well; however, this question did present some nice alternatives. – Jed Smith Oct 26 '09 at 14:39
  • 3
    If the actual list had None somewhere in the middle, your last function would remove those as well. – cnu Oct 10 '11 at 05:11
  • This makes an assumption about the implementation of zip_longest (ie: that it visits the iterables strictly in order in a loop) which might not strictly be safe. It's probably safe to use, but a bit of a smell. – Emlyn O'Regan Feb 08 '16 at 05:08
14

Itertools.groupby is a fine tool, here is a way to split a list of integers simply by using integer division:

>>> for key, group in itertools.groupby(range(10), lambda k: k//3):
...  print key, list(group)
... 
0 [0, 1, 2]
1 [3, 4, 5]
2 [6, 7, 8]
3 [9]

(The list has to start with 0 to begin with a full group.)

u0b34a0f6ae
  • 48,117
  • 14
  • 92
  • 101
  • 2
    Your code _only_ works on `range(i)` where i is some integer, not for "a list of arbitrary length" as specified. Your answer is not correct. – Tom Swirly Apr 20 '19 at 15:38
11
n = 25    
list_of_lists = [L[i:i+n] for i in range(0, len(L), n)]

it gives you the list of lists [[0..24], [25..49], ..]

If len(L) % n isn't 0, the last element's (list_of_lists[-1]) lenght will be len(L) % n.

kender
  • 85,663
  • 26
  • 103
  • 145
0

Here is recursion version. It is inefficient because Python has recursion limits, but this version illustrates that every task can be solved through recursion.

def split_to_groups(l, n):
    assert (len(l) / n) < 998, "Can't split to {} groups".format(len(l) / n)
    if l == []:
        return []
    else:
        f = [l[:n]]
        f.extend(split_to_groups(l[n:], n))
        return f
Dmitri
  • 121
  • 1
  • 7