4

How can I yield multiple items at a time from an iterable object?

For example, with a sequence of arbitrary length, how can I iterate through the items in the sequence, in groups of X consecutive items per iteration?

Matt Joiner
  • 112,946
  • 110
  • 377
  • 526
  • Straight duplicate of http://stackoverflow.com/questions/312443/how-do-you-split-a-list-into-evenly-sized-chunks-in-python –  Feb 05 '10 at 00:40
  • @Paul: Not a duplicate, this uses "iterable" not "list", and quoting Ned from there: "That's an interesting extension to the question, but the original question clearly asked about operating on a list." –  Feb 05 '10 at 04:25
  • The highest rated answer on the link above is the same as the one given here. And the same as the approved answer on this other duplicate: http://stackoverflow.com/questions/434287/what-is-the-most-pythonic-way-to-iterate-over-a-list-in-chunks When the best answer is a 4-line function copied from the standard library, the question adds little of value. –  Feb 05 '10 at 06:22
  • I've seen answers extracted from elsewhere when they have great value several times on SO. Solutions for iterator grouping have been mentioned in passing on a few questions, but no direct question relating to it has been asked. I've put this question here purely so the next poor sod who comes along with the same problem immediately finds the right answer. – Matt Joiner Feb 05 '10 at 11:06
  • I came across a little gruff; it was not pleasant seeing someone else smear my name across their question like this, and I had to step back a bit to see that was the cause. –  Feb 09 '10 at 23:36

2 Answers2

7

Your question is a bit vague, but check out the grouper recipe in the itertools documentation.

def grouper(n, iterable, fillvalue=None):
    "grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"
    args = [iter(iterable)] * n
    return izip_longest(fillvalue=fillvalue, *args)

(Zipping the same iterator several times with [iter(iterable)]*n is an old trick, but encapsulating it in this function avoids confusing code, and it is the same exact form and interface many people will use. It's a somewhat common need and it's a bit of a shame it isn't actually in the itertools module.)

Mike Graham
  • 73,987
  • 14
  • 101
  • 130
  • 2
    AFAIK `[iter(iterable)]*n` is dirty because the execution order of function arguments is not defined, i.e. you cannot be sure that the first argument of a function get evaluated first. This might not be a problem as long as you use the same interpreter but may produce weird results with other interpreters. – Michael Feb 04 '10 at 21:51
  • 1
    @Michael, the order is guaranteed, see here: http://docs.python.org/library/itertools.html#itertools.izip – Matt Joiner Feb 04 '10 at 22:00
  • @Michael, I don't see why you would think that, order is quite well-defined. – Mike Graham Feb 04 '10 at 23:28
  • Right, that's not a problem here. What I meant: When you write something like `f(3+4, 5+6)` then it's not defined whether `3+4` or `5+6` is evaluated first. But that's not a problem here because both arguments are the same and `iter` gets evaluated before. – Michael Feb 05 '10 at 00:03
  • 1
    @Michael: The evaluation order is guaranteed. `f(x,y)` will always evaluate `x` before `y`. See http://docs.python.org/reference/expressions.html#evaluation-order. – unutbu Feb 05 '10 at 01:17
  • Hm, didn't know that one, looks like both are guaranteed. – Matt Joiner Feb 05 '10 at 02:15
  • Of course, with the particular example of `f(3+4, 5+6)` they may well be constantfolded and the expressions not evaluated anyhow... – Mike Graham Feb 05 '10 at 18:00
2

Here's another approach that works on older version of Python that don't have izip_longest:

def grouper(n, seq):
  result = []
  for x in seq:
    result.append(x)
    if len(result) >= n:
      yield tuple(result)
      del result[:]
  if result:
    yield tuple(result)

No filler, so the last group might have fewer than n elements.

Laurence Gonsalves
  • 137,896
  • 35
  • 246
  • 299
  • 1
    Note that all the functions in `itertools` have pure Python implementations in the documentation if someone is on a Python older then 2.6. – Mike Graham Feb 04 '10 at 20:42