10

Let's say I have a generator like this:

def a(data):
    for val in data:
        yield val

And let's say I want to wrap this generator in another generator, b, that only yields some of the values from a, depending on their value. b should be able to forward values sent back from the caller to a. I know that the most current way of wrapping a generator in another generator is to use the yield from statement. Something like:

def b(data):
    yield from val = a(data) if val == "foo"

I know that syntax is wrong (it's just to get across the idea), so I'm wondering if there is a correct way to make yield from work with a conditional statement. Or is there some other construct I should use?

LateCoder
  • 2,163
  • 4
  • 25
  • 44

3 Answers3

6

As you said, yield from is only for the case where you want to pass everything from the wrapped generator through. If you don't want that, you need to manually iterate over the wrapped generator and do what you want:

def b(data):
    for value in a(data):
        if some_condition(value):
            yield value
        # otherwise don't yield it.

If you want to handle send, you need to handle that yourself too. Here is a simple example that you can play around with to see what's going on:

def a():
    sent = yield "Begin"
    for x in [1, 2, 3]:
        if sent:
            sent = yield "You sent {0}".format(sent)
        else:
            sent = yield x

def b():
    gen = a()
    val = yield next(gen)
    while True:
        wrappedVal = gen.send(val)
        if wrappedVal == 2:
            continue
        val = yield wrappedVal

Basically in this example b "hides" the value 2 if it is yielded by a. (I adapted this from my answer to this question, which is similarly about wrapping generators, but in a slightly different way. You may find discussion there useful, though.)

However, you would need to be very careful when doing this. Since b may skip values from a, any caller that tries to send values into b may not get the expected results, because b may skip values in unexpected places. In cases where you're using send, it often means the caller watches the generator for certain yielded values that communicate some information, then sends values back in response. With b in the middle, this communication may get out of sync. For instance, suppose a yields a "message" which b yields through to the caller, and the caller sends a response back, which b then sends to a. But suppose b swallows a's response. This will cause problems for the caller. You may still be able to make it work, but it just means you have to be very careful in writing b to main any communication channels that are expected by the API of a.

Community
  • 1
  • 1
BrenBarn
  • 242,874
  • 37
  • 412
  • 384
  • How would I pass values that are sent into `b` back to `a`, though? – LateCoder Dec 03 '15 at 20:35
  • @LateCoder: Things get even more manual; you have to manually call `send` and catch `StopIteration`. Also, you need to decide what to send to `a` when `b` skips a value. – user2357112 Dec 03 '15 at 20:41
  • @LateCoder: I updated my answer. But you need to be careful about using `send` if `b` can skip values, because it may mean that values sent by the caller to `b` are not passed to `a` at "the right time", or that values from `a` do not reach `b`'s caller at "the right time". – BrenBarn Dec 03 '15 at 20:51
  • Thank you! I put up the final answer I came to, which seems to be working, inspired by your answer. – LateCoder Dec 03 '15 at 21:03
3

Assuming function a returns an Iterator[str], how about this?

yield from filter(lambda x: x == "foo" , a(data))
Abhijit Sarkar
  • 21,927
  • 20
  • 110
  • 219
  • 1
    This is a great answer for cases where you don't need for all the features of `yield from` (forwarding `.send`, `.throw`, and `.close` calls). – mtraceur Nov 25 '22 at 17:29
  • @mtraceur based on the question posted, is there any reason to assume any of the other features you mentioned are relevant to the discussion? – Abhijit Sarkar Nov 26 '22 at 00:40
  • Yes. 1st of all that should be the default assumption - if someone asks "is there a way to get `yield from`+{other behavior}", the presumably want `yield from` behavior, not just *parts* of `yield from` behavior. 2nd, the asker both posted their own answer and asked a comment on another answer about forwarding sent values. 3rd, a question like this is very likely to be found by people who would benefit from having their attention drawn to the absence of send/throw/close forwarding. – mtraceur Nov 27 '22 at 07:13
  • 1
    @mtraceur The way stackoverflow works, you’re free to write your own answer if you think you’ve a better one than lecturing on others’ answers about supposed faults. So, go ahead, and do share your knowledge. – Abhijit Sarkar Nov 27 '22 at 15:07
  • The way StackOverflow works, we also add comments to suggest improvements or provide relevant clarifying information (and this is helpful independently of posting answers, which often take more time and preparation - for example, in this case, the best answer I could give only becomes possible after I finish a certain open source library which would include a `filter` implementation that does full generator delegation). Anyway, I encourage you to try re-reading my comments less negatively - my first comment was mostly a compliment and my second comment was answering *a question you asked*. – mtraceur Nov 28 '22 at 03:18
0

Since it looks like I can't incorporate a conditional onto a yield from, I solved this problem by handling the cases I cared about manually. This is what I ended up with: 

def a():
    data = get_data_somewhere()
    for val in data:
        yield val

def b():
    a_gen = a()
    val = next(a_gen)
    try:
        while True:
            if val == "foo":
                received = yield val
                val = a_gen.send(received)
            else:
                val = a_gen.send(None)
     except StopIteration:
         pass
     finally:
         del a_gen
LateCoder
  • 2,163
  • 4
  • 25
  • 44