16

I noticed that itertools does not (it seems to me) have a function capable of interleaving elements from several other iterable objects (as opposed to zipping them):

def leaf(*args): return (it.next() for it in cycle(imap(chain,args)))
tuple(leaf(['Johann', 'Sebastian', 'Bach'], repeat(' '))) => ('Johann', ' ', 'Sebastian', ' ', 'Bach', ' ')

(Edit) The reason I ask is because I want to avoid unnecessary zip/flatten occurrences.

Obviously, the definition of leaf is simple enough, but if there is a predefined function that does the same thing, I would prefer to use that, or a very clear generator expression. Is there such a function built-in, in itertools, or in some other well-known library, or a suitable idiomatic expression?

Edit 2: An even more concise definition is possible (using the functional package):

from itertools import *
from functional import *

compose_mult = partial(reduce, compose)
leaf = compose_mult((partial(imap, next), cycle, partial(imap, chain), lambda *args: args))
Marcin
  • 48,559
  • 18
  • 128
  • 201
  • 8
    In a totally unrelated note I can't help but mention that the Bach in question was Johann, not John. – 9000 Jan 07 '12 at 13:35
  • @9000: Quite - this fell out of a unit test where I absentmindedly wrote it as it is – Marcin Jan 07 '12 at 13:59

3 Answers3

16

You're looking for the built-in zip and itertools.chain.from_iterable to flatten the result:

>>> import itertools
>>> list(zip(['Johann', 'Sebastian', 'Bach'], itertools.repeat(' ')))
[('Johann', ' '), ('Sebastian', ' '), ('Bach', ' ')]
>>> list(itertools.chain.from_iterable(_))
['Johann', ' ', 'Sebastian', ' ', 'Bach', ' ']

Note that I used list just to force a nice output. Using the standard itertools, alternative implementations for leaf would be:

leaf = lambda *a: itertools.chain.from_iterable(itertools.izip(*a)) # Python 2.x
leaf = lambda *a: itertools.chain.from_iterable(zip(*a))            # Python 3.x
phihag
  • 278,196
  • 72
  • 453
  • 469
  • @larsmans Safer in what way? `itertools.izip` has been removed from Python since 3.0. – phihag Jan 07 '12 at 13:05
  • 1
    Okay. I still live in the Python 2.x world mostly. – Fred Foo Jan 07 '12 at 13:24
  • Right, but that still introduces a zip, and then flattens the zipped pairs. My whole motivation (which I should have stated) was to avoid doing that. – Marcin Jan 07 '12 at 13:24
  • 2
    @Marcin What is your problem with zipping and then flattening? – phihag Jan 07 '12 at 15:20
  • 3
    Speaking for @Marcin, imagine that the input iterables are huge or infinite and he'd probably rather process them iteratively rather than all at once. – Kirk Strauser Jan 07 '12 at 15:39
  • @KirkStrauser Huh? Both `zip` (in Python 2 `itertools.izip`) and `chain.from_iterable` return generators and work fine with huge or infinite inputs. – phihag Jan 07 '12 at 16:04
  • @ phihag @KirkStrauser: The answer is that it's an unnecessary set of operations (boxing and unboxing in the tuples), and in larger expressions, it makes one more thing to understand. – Marcin Jan 07 '12 at 16:59
  • @Marcin Sorry, I don't understand what you mean with *boxing and unboxing*. The memory consumption of the solution is in O(1), so it can't be the allocation of memory. – phihag Jan 08 '12 at 12:17
  • @phihag: Are you really saying that zipping involves no memory allocation? – Marcin Jan 08 '12 at 12:53
  • @phihag: Even if a sufficiently optimising compiler is able to consume only one tuple's worth of memory, there will be an unnecessary roundtrip through the tuple constructor, unless the compiler is able to completely optimise it away. – Marcin Jan 08 '12 at 13:02
8

The itertools roundrobin() recipe would've been my first choice, though in your exact example it would produce an infinite sequence, as it stops with the longest iterable, not the shortest. Of course, it would be easy to fix that. Maybe it's worth checking out for a different approach?

Matt Luongo
  • 14,371
  • 6
  • 53
  • 64
  • That's the sort of thing I was thinking, except that (a) I prefer the leaf semantics (b) my definition is more concise. – Marcin Jan 07 '12 at 17:01
2

This custom function interleaves iterators and continues untill every iterator is exausted

def interleave_iterators(*iterators):
    finished = [False for x in range(len(iterators))]
    stop_cond = functools.reduce(lambda x,y:not x or not y,finished)
    while stop_cond:
        for i,it in enumerate(iterators):
            try:
                yield next(it)
            except StopIteration:
                finished[i] = True
        stop_cond = functools.reduce(lambda x,y:not x or not y,finished)

For example

it1 = iter([2,4,6,8])
it2 = iter([1,3,5,7,9,11,13])
for x in interleave_iterators(it1,it2): # prints 2 1 4 3 6 5 8 7 9 11 13
    print(str(x),end=" ")
Dema
  • 301
  • 2
  • 5