Transpose nested generators

Question

Is there a clear way to iterate over items for each generator in a list? I believe the simplest way to show the essence of the question is o proved an expample. Here it is

0. Assume we have an function returning generator:

def gen_fun(hint):
    for i in range(1,10):
        yield "%s %i" % (hint, i)

1. Clear solution with straight iteration order:

hints = ["a", "b", "c"]
for hint in hints:
    for  txt in gen_fun(hint):
        print(txt)

This prints

a 1
a 2
a 3
...
b 1
b 2
b 3
...

2. Cumbersome solution with inverted iterating order

hints = ["a", "b", "c"]
generators = list(map(gen_fun, hints))
any = True
while any:
    any = False
    for g in generators:
        try:
            print(next(g))
            any = True
        except StopIteration:
            pass

This prints

a 1
b 1
c 1
a 2
b 2
...

This works as expected and does what I want.

Bonus points:

The same task, but gen_fun ranges can differ, i.e

def gen_fun(hint):
    if hint == 'a':
        m = 5
    else:
        m = 10
    for i in range(1,m):
        yield "%s %i" % (hint, i)

The correct output for this case is:

a 1
b 1
c 1
a 2
b 2
c 2
a 3
b 3
c 3
a 4
b 4
c 4
b 5
c 5
b 6
c 6
b 7
c 7
b 8
c 8
b 9
c 9

The querstion:

Is there a way to implement case 2 cleaner?

why do you call list on map? In python2 it is a list, and in python3 it's still iterable — en_Knight, Nov 13 '15 at 18:59
@en_Knight, in python3 you can only follow it once, but I have to follow it repeatedly. I tried without list first and it produced sort of unexpected results. Try it! — Lol4t0, Nov 13 '15 at 19:00
Oh I see, that makes sense. And I see what you mean by inverse order, yurib's answer looks good — en_Knight, Nov 13 '15 at 19:01
@en_Knight, If you have a better definition of what I meant by inverted order, you are welcome :) — Lol4t0, Nov 13 '15 at 19:05
I don't see *any* "inversion" in any iteration order. What are you talking about? To me inversion == `reversed` or something like that. — Bakuriu, Nov 13 '15 at 19:08
@Bakuriu If he's asking for a better implementation of case 2, it is clear that the expected output is obtained by running the code. — eguaio, Nov 13 '15 at 19:10
@eguaio There's no "inversion" of any kind there. I suggest the Op edit his question removing references to things that can only confuse readers. Also, I don't see why we should be forced to run the code to obtain the expected output... he could copy & paste the output for the example he provided, making things clearer even to people who don't have a python installation (e.g. people on mobile). — Bakuriu, Nov 13 '15 at 19:12
@ If you have a better definition of what I meant by inverted order, you are welcome. — Lol4t0, Nov 13 '15 at 19:14
I agree that it's clear from running the code what you mean, but it should probably be clear from the question too. How about "iterate over items for each generator in a list; traverse generator-first not element-first" It's like a depth-wise vs. a breadth-wise search. I also agree that it would be helpful had you put what the output was in each case, not that it's too difficult to run — en_Knight, Nov 13 '15 at 19:15
Ok, I though of math term `Transpose`. It can be applied here: `Transpose nested generators` may be? I refuse to accept that you can mentally understand what `iterating over items for each generator in a list` is — Lol4t0, Nov 13 '15 at 19:23
@en_Knight, for the auestion body your definition is ok I believe but I don't like it in a title :( — Lol4t0, Nov 13 '15 at 19:26

score 3 · Accepted Answer · edited May 23 '17 at 12:23

3

If i understand the question correctly, you can use zip() to achieve the same thing as that whole while any loop:

hints = ["a", "b", "c"]
generators = list(map(gen_fun, hints))
for x in zip(*generators):
    for txt in x:
        print(txt)

output:

a 1
b 1
c 1
a 2
b 2
...

UPDATE:

If the generators are of different length, zip 'trims' them all to the shortest. you can use itertools.izip_longest (as suggested by this q/a) to achieve the opposite behaviour and continue yielding until the longest generator is exhausted. You'll need to filter out the padded values though:

hints = ["a", "b", "c"]
generators = list(map(gen_fun, hints))
for x in zip_longest(*generators):
    for txt in x:
        if txt:
            print(txt)

edited May 23 '17 at 12:23

Community

1
1

answered Nov 13 '15 at 19:01

yurib

8,043
3
30
55

@en_Knight It's inverse compared to solution 1. i.e. hints are in the nested loop. – yurib Nov 13 '15 at 19:04
Woh! That doesn't work if gen_fun function differ . Overlooked that, sorry – Lol4t0 Nov 13 '15 at 19:44
See edit. (Though I'm really sorry I overlooked that in the first place) – Lol4t0 Nov 13 '15 at 19:48
It should be `itertools.zip_longest` (without `i`) in python 3, but otherwise that's perfect! Thanks again – Lol4t0 Nov 13 '15 at 19:57

score 0 · Answer 2 · answered Nov 13 '15 at 19:44

You might want to look into itertools.product:

from itertools import product

# Case 1
for tup in product('abc', range(1,4)):
    print('{0} {1}'.format(*tup))

print '---'

# Case 2
from itertools import product
for tup in product(range(1,4), 'abc'):
    print('{1} {0}'.format(*tup))

Output:

a 1
a 2
a 3
b 1
b 2
b 3
c 1
c 2
c 3
---
a 1
b 1
c 1
a 2
b 2
c 2
a 3
b 3
c 3

Note that the different between case 1 and 2 are just the order of parameters passed into the product function and the print statement.

well, apparently I'm doing some more complicated task in `gen_fun`, so you cannot just strike it out — Lol4t0, Nov 13 '15 at 19:52