Python idiom to chain (flatten) an infinite iterable of finite iterables?

Question

Suppose we have an iterator (an infinite one) that returns lists (or finite iterators), for example one returned by

infinite = itertools.cycle([[1,2,3]])

Note: it is not the same problem as just flattening a nested list - solutions which try to exhaustively process the list will not work, e.g. [item for sublist in infinite for item in sublist] will exhaust memory.

What is a good Python idiom to get an iterator (obviously infinite) that will return each of the elements from the first iterator, then each from the second one, etc. In the example above it would return 1,2,3,1,2,3,.... The iterator is infinite, so itertools.chain(*infinite) will not work.

Related

• Flattening a shallow list in python

Answer 1

Starting with Python 2.6, you can use itertools.chain.from_iterable:

itertools.chain.from_iterable(iterables)

You can also do this with a nested generator comprehension:

def flatten(iterables):
    return (elem for iterable in iterables for elem in iterable)

Answer 2

Use a generator:

(item for it in infinite for item in it)

The * construct unpacks into a tuple in order to pass the arguments, so there's no way to use it.