Yield statement's effect on program flow

Question

I'm trying to understand the use of the yield keyword in C#, as a queue modelling package I'm using makes extensive use of it.

To demonstrate the use of yield, I am playing around with the following code:

using System;
using System.Collections.Generic;
public class YieldTest
{
    static void Main()
    {
        foreach (int value in ComputePower(2, 5))
        {
            Console.Write(value);
            Console.Write(" ");
        }
        Console.WriteLine();
    }
    /**
     * Returns an IEnumerable iterator of ints
     * suitable for use in a foreach statement
     */
    public static IEnumerable<int> ComputePower(int number, int exponent)
    {
        Console.Write ("Arguments to ComputePower are number: " + number + " exponent: " + exponent + "\n");
        int exponentNum = 0;
        int numberResult = 1;
        while (exponentNum < exponent)
        {
            numberResult *= number;
            exponentNum++;
            // yield: 
            // a) returns back to the calling function (foreach),
            // b) updates iterator value (2,4,8,16,32 etc.)
            yield return numberResult;
        }
    }
}

It's pretty evident what the code does, it simply raises 2 to a power using ComputePower which returns an IEnumerable. In debugging the code, I see the yield statement return control to the foreach loop, and the value variable is updated with the latest result of the power ie. 2, 4, 8, 16, 32.

Not fully understanding the use of yield, I expected ComputePower to be called a number of times as value iterates through ComputePower and that I would see the "Arguments to ComputePower are " etc. console write occur 5 times. What actually happens though is it seems the ComputePower method is called only once. I see the "Arguments to ComputePower.." string only once per run.

Can somebody explain why this is the case? Does it have something to do with the yield keyword?

Answer 1

The foreach will iterate the IEnumerable returned from ComputePower. "Yield return" automatically creates an implementation of IEnumerable so you don't have to hand-roll it. If you put a breakpoint inside your "while"-loop you will see it getting called for each iteration

From msdn:

You consume an iterator method by using a foreach statement or LINQ query. Each iteration of the foreach loop calls the iterator method. When a yield return statement is reached in the iterator method, expression is returned, and the current location in code is retained. Execution is restarted from that location the next time that the iterator function is called.

Answer 2

yield return causes the compiler to build a state machine which implements IEnumerable<T> using the body of your method. It returns an object from your method without actually invoking the body of your method as you wrote it - the compiler has replaced it with something more complex.

When you call MoveNext() on the IEnumerator<T> produced by the state machine (such as during a foreach loop), the state machine executes your method code until it reaches the first yield return statement. It then sets the value of Current to whatever value you returned and then yields control back to the caller.

In practise, it looks as though your method body is executed once per iteration and the loop is "interrupted" each time you reach a yield return statement.

If you put a breakpoint in your method's while loop, you will see the stack contains a call to MoveNext() on the compiler-generated type which your method's body has become part of.

Answer 3

On a high level, you can think of yield as saying 'return a value and freeze the current state of the method. When the generator is next called, the method will thaw and resume starting at the line following the yield'. So any line that is only at the start of the method and not actually in the loop where yield exists will only be called once, it doesn't start the whole method over again.

On a low level, yield is implemented by the compiler transforming your method into a state machine, where a jump table is added at the start of the method and which jump we take (which yield line of code we start executing at when you call the method) is determined by the 'state' the generator was last in. A similar coding technique is used for await/async state machines, and allows a lot of complexity to be hidden from the programmer under an easier to understand model.

Answer 4

The yield operator will force compiler to create a custom class which will implement your logic. The better way to understand it is decompile result exe and watch into it.