12

In Python most examples of yield from explain it with saying that 

yield from foo()

is similar to 

for x in foo(): yield x

On the other hand it doesn't seem to be exactly the same and there's some magic thrown in. I feel a bit uneasy about using a function that does magic that I don't understand. What do I have to know about the magic of yield from to avoid getting into a situation where the magic does something I don't expect? What advantages does the magic provide, that I should be aware of?

Christian
  • 25,249
  • 40
  • 134
  • 225

1 Answers1

13

When foo() returns a regular iterable, the two are equivalent. The 'magic' comes into play when foo() is a generator too. At that moment, the yield from foo() and for x in foo(): yield x cases differ materially.

A generator can be sent data too, using the generator.send() method. When you use the for loop, the yield x expression 'receives' the sent data; the foo() generator will never see this. But when you use yield from the sent data goes straight to whatever yield expression the delegated-to generator is currently paused at. In other words, yield from passes on the sent data so the delegated-to generator can receive it instead.

You can also raise exceptions in a generator, with generator.throw(); with the for loop case, the exception is raised from the yield x line, while with yield from the exception is passed on again; the exception is raised inside foo() instead.

Together, this means that yield from in essence replaces the current generator for the duration of the delegated iteration.

The delegated-to generator also gets to communicate with the parent generator, when done the .value attribute of the StopIteration exception raised is returned as the value of the yield from expression. You can set the value of that exception by using return <expression> in the delegated-to foo() generator, or you can use raise StopIteration(<expression>) explicitly.

yield from was introduced into the language with PEP 380: Syntax for Delegating to a Subgenerator.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 1
    +1 for the simple explanation. `yield from` allows you to refactor a generator into several functions without changing the behavior of the code (though the nested generators are still not optimized as far as I know e.g., `O(n*k)` behavior is possible if you are implementing a power set instead of `O(n)` where `n=2**k`). – jfs Sep 22 '14 at 12:33
  • This really set the foundation of your answer: _"When `foo()` returns a regular iterable, the two are equivalent. The 'magic' comes into play when `foo()` is a generator too."_ Wanna delegate to another generator? Use `yield from`. – sargas Apr 27 '15 at 23:01