6

Ok, as I was poking around with building a custom enumerator, I had noticed this behavior that concerns the yield

Say you have something like this:

  public class EnumeratorExample 
  {

        public static IEnumerable<int> GetSource(int startPoint) 
        {
                int[] values = new int[]{1,2,3,4,5,6,7};
                Contract.Invariant(startPoint < values.Length);
                bool keepSearching = true;
                int index = startPoint;

                while(keepSearching) 
                {
                      yield return values[index];
                      //The mind reels here
                      index ++ 
                      keepSearching = index < values.Length;
                }
        }

  } 

What makes it possible underneath the compiler's hood to execute the index ++ and the rest of the code in the while loop after you technically do a return from the function?

dexter
  • 7,063
  • 9
  • 54
  • 71

5 Answers5

9

The compiler rewrites the code into a state machine. The single method you wrote is split up into different parts. Each time you call MoveNext (either implicity or explicitly) the state is advanced and the correct block of code is executed.

Suggested reading if you want to know more details:

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
Mark Byers
  • 811,555
  • 193
  • 1,581
  • 1,452
  • Yes, ok, state machine, that's what I read. But what kinda code does it generate and whats that state machine is doing with it? Pseudo code would be greatly appreciated. – dexter Dec 15 '10 at 22:58
  • @Max Malygin: The article I linked to http://csharpindepth.com/Articles/Chapter6/IteratorBlockImplementation.aspx shows the code that is generated. – Mark Byers Dec 15 '10 at 23:05
  • Also check Matt Greer's answer: http://stackoverflow.com/questions/4455796/the-wonders-of-the-yield-keyword/4455832#4455832 He also found part 4 of the series. – Mark Byers Dec 15 '10 at 23:11
  • Thanks for the shout-out. This link might be better, since it gives them in order: http://blogs.msdn.com/b/ericlippert/archive/tags/iterators/ – Eric Lippert Dec 16 '10 at 04:00
4

The compiler generates a state-machine on your behalf.

From the language specification:

10.14 Iterators

10.14.4 Enumerator objects

When a function member returning an enumerator interface type is implemented using an iterator block, invoking the function member does not immediately execute the code in the iterator block. Instead, an enumerator object is created and returned. This object encapsulates the code specified in the iterator block, and execution of the code in the iterator block occurs when the enumerator object’s MoveNext method is invoked. An enumerator object has the following characteristics:

• It implements IEnumerator and IEnumerator, where T is the yield type of the iterator.

• It implements System.IDisposable.

• It is initialized with a copy of the argument values (if any) and instance value passed to the function member.

• It has four potential states, before, running, suspended, and after, and is initially in the before state.

An enumerator object is typically an instance of a compiler-generated enumerator class that encapsulates the code in the iterator block and implements the enumerator interfaces, but other methods of implementation are possible. If an enumerator class is generated by the compiler, that class will be nested, directly or indirectly, in the class containing the function member, it will have private accessibility, and it will have a name reserved for compiler use (§2.4.2).

To get an idea of this, here's how Reflector decompiles your class:

public class EnumeratorExample
{
    // Methods
    public static IEnumerable<int> GetSource(int startPoint)
    {
        return new <GetSource>d__0(-2) { <>3__startPoint = startPoint };
    }

    // Nested Types
    [CompilerGenerated]
    private sealed class <GetSource>d__0 : IEnumerable<int>, IEnumerable, IEnumerator<int>, IEnumerator, IDisposable
    {
        // Fields
        private int <>1__state;
        private int <>2__current;
        public int <>3__startPoint;
        private int <>l__initialThreadId;
        public int <index>5__3;
        public bool <keepSearching>5__2;
        public int[] <values>5__1;
        public int startPoint;

        // Methods
        [DebuggerHidden]
        public <GetSource>d__0(int <>1__state)
        {
            this.<>1__state = <>1__state;
            this.<>l__initialThreadId = Thread.CurrentThread.ManagedThreadId;
        }

        private bool MoveNext()
        {
            switch (this.<>1__state)
            {
                case 0:
                    this.<>1__state = -1;
                    this.<values>5__1 = new int[] { 1, 2, 3, 4, 5, 6, 7 };
                    this.<keepSearching>5__2 = true;
                    this.<index>5__3 = this.startPoint;
                    while (this.<keepSearching>5__2)
                    {
                        this.<>2__current = this.<values>5__1[this.<index>5__3];
                        this.<>1__state = 1;
                        return true;
                    Label_0073:
                        this.<>1__state = -1;
                        this.<index>5__3++;
                        this.<keepSearching>5__2 = this.<index>5__3 < this.<values>5__1.Length;
                    }
                    break;

                case 1:
                    goto Label_0073;
            }
            return false;
        }

        [DebuggerHidden]
        IEnumerator<int> IEnumerable<int>.GetEnumerator()
        {
            EnumeratorExample.<GetSource>d__0 d__;
            if ((Thread.CurrentThread.ManagedThreadId == this.<>l__initialThreadId) && (this.<>1__state == -2))
            {
                this.<>1__state = 0;
                d__ = this;
            }
            else
            {
                d__ = new EnumeratorExample.<GetSource>d__0(0);
            }
            d__.startPoint = this.<>3__startPoint;
            return d__;
        }

        [DebuggerHidden]
        IEnumerator IEnumerable.GetEnumerator()
        {
            return this.System.Collections.Generic.IEnumerable<System.Int32>.GetEnumerator();
        }

        [DebuggerHidden]
        void IEnumerator.Reset()
        {
            throw new NotSupportedException();
        }

        void IDisposable.Dispose()
        {
        }

        // Properties
        int IEnumerator<int>.Current
        {
            [DebuggerHidden]
            get
            {
                return this.<>2__current;
            }
        }

        object IEnumerator.Current
        {
            [DebuggerHidden]
            get
            {
                return this.<>2__current;
            }
        }
    }
}
Ani
  • 111,048
  • 26
  • 262
  • 307
2

Yield is magic.

Well, not really. The compiler generates a full class to generate the enumeration that you're doing. It's basically sugar to make your life simpler.

Read this for an intro.

EDIT: Wrong this. Link changed, check again if you have once.

Donnie
  • 45,732
  • 10
  • 64
  • 86
  • @Max - Depending on when you clicked through on the link, it may be different now. I originally posted the wrong one. – Donnie Dec 15 '10 at 23:01
2

Here is an excellent blog series (from Microsoft veteran Raymond Chen) that details how yield works:

Glorfindel
  • 21,988
  • 13
  • 81
  • 109
Matt Greer
  • 60,826
  • 17
  • 123
  • 123
2

That's one of the most complex parts of the C# compiler. Best read the free sample chapter of Jon Skeet's C# in Depth (or better, get the book and read it :-)

Implementing iterators the easy way

For further explanations see Marc Gravell's answer here:

Can someone demystify the yield keyword?

Community
  • 1
  • 1
Dirk Vollmar
  • 172,527
  • 53
  • 255
  • 316