26

According to Eric Lippert, anonymous iterators were not added to the language because it would be overly complicated to implement it.

That is okay with me, and it didn't bother me until they went ahead and implemented anonymous asynchronous methods. The compiler has to do the same thing for async methods as it has to for iterators (convert them into state machines), so I am very confused why anonymous iterators are not allowed as well, when anonymous async methods are.

Can someone shed some light on this?

svick
  • 236,525
  • 50
  • 385
  • 514
jakobbotsch
  • 6,167
  • 4
  • 26
  • 39
  • 1
    In addition to iterator lambdas, there could also be iterator expressions. This feature could look like this: `IEnumerable e = enum { yield return 1; };`. That would make argument validation much simpler because you don't need to extract a 2nd method. – usr Dec 31 '13 at 20:39
  • 2
    Though I am happy to shed some light, I note that this is not a *specific* question about *actual code*, and that it is in fact a historical question about the motivations of the design and implementation teams only answerable by someone on those teams. This kind of question is a poor fit for StackOverflow. – Eric Lippert Jan 01 '14 at 00:14

3 Answers3

26

According to Eric Lippert, anonymous iterators were not added to the language because it would be overly complicated to implement it.

That is not precisely what I intended to convey. The relevant cost is implementation cost, yes, but it is implementation cost in an existing compiler which was not set up architecturally to implement that complex feature.

The compiler has to do the same thing for async methods as it has to for iterators (convert them into state machines), so I am very confused why anonymous iterators are not allowed as well, when anonymous async methods are.

A brief history is relevant. C# first had anonymous methods and iterator blocks in C# 2.0. When I added lambdas in C# 3.0 it was a major cost to refactor all of the existing anonymous method code so that it could handle all the new features of lambdas. That made it even more complicated and expensive to modify. Making iterator block lambdas was judged too costly for the benefits that would be accrued; it would have been a large percentage of the total cost. We could not afford it. If you added up every team in Developer Division's work schedule, the team with the "longest pole" was the C# 3.0 compiler team, and my work on the semantic analyzer was IIRC the longest pole on the compiler team. Every day we might have slipped C# 3.0, that would have been a day that Visual Studio would have slipped. Therefore anything that didn't make LINQ better was cut, and that included iterator lambdas.

In C# 4, iterator lambdas were one feature of many that were considered. We had a list of potential good features literally longer than your arm and we could afford to do less than a tenth of them.

In C# 5 the team added async methods. The design and implementation teams tried for a long time to come up with an underlying abstraction that was common to both the iterator block and await rewrites; they are obviously similar, as you note. But ultimately, the cost of finding the general solution did not pay for itself. Generality is surprisingly expensive, and finding a generality that by design unifies only two things is silly if it is not cheap.

Therefore the decision was made to implement the await rewriter as its own thing. Given that the team was going to take on this large cost, and given that the original transformation of async methods was going to be into a lambda form anyway, the decision was made to invest in the full feature: async methods containing lambdas, async lambdas containing lambdas, the whole deal. The cost of that feature was a small fraction of the cost of the whole feature, which was extremely expensive.

And again, we have a problem with long poles. Any work on the lambda engine that could potentially have destabilized await is to be avoided, and that includes trying to make them work with iterator blocks.

Now compare Visual Basic. VB for a long time had no iterator blocks at all. When they were added, there was no existing infrastructure to keep working! The whole thing could be built from the ground up to handle iterator blocks containing lambdas and lambdas containing iterator blocks, and so that was done.

The C# compiler has been thoroughly rearchitected and rewritten via the Roslyn project. I am hoping that this will lower the cost of implementing iterator block lambdas in a hypothetical future version of C#. We shall see!

Eric Lippert
  • 647,829
  • 179
  • 1,238
  • 2,067
  • I am only aware of the high level transformation the C# compiler does (iterator block -> state machine, async method -> state machine), so that is why I was under the assumption that generalizing this would not be complicated. My understanding from your answer is that there are a lot of slight differences and implementation details between the two that makes it much harder to make a generalized solution, and that is exactly the answer I was looking for. – jakobbotsch Jan 01 '14 at 14:23
  • 1
    @Janiels: There are aspects other than difficulty in adapting the infrastructure to handling two kinds of state machines. For example, suppose tomorrow the team decides that the restriction on an await in a catch can be overcome through a tricky change to the code. Now we have a problem. Yield return is not legal in a catch. Either they get lucky and the change *enables* yield return in catch as well without breaking anything, or the change *destabiizes* the existing code that handles yield returns inside try structures. The risk of the latter is high if you have a general solution. – Eric Lippert Jan 01 '14 at 16:29
  • 2
    @Janiels: In short: code re-use is actually seldom the savings that you think it will be. Often it is a better use of limited resources to make two similar things that each do one thing well, and can be modified at will without affecting the other than it is to make one thing that does two things adequately. The Roslyn team had this debate for literally months in the large: should we have one compiler that can compile both C# and VB, or two compilers that each compile one language well *and can be changed in the future independently*? We chose the latter. – Eric Lippert Jan 01 '14 at 16:32
  • @EricLippert, I don't see why it couldn't do a translation instead? How is **giving those unnamed iterators a randomized name** unimplementable? – Pacerier Mar 20 '17 at 10:09
5

Anonymous iterator blocks, while nice, do not have a particularly compelling benefit. It is not a huge deterrent for iterator blocks to be refactored into their own method.

async anonymous methods make a lot more conceptual sense, don't warrant refactoring into their own method quite the same way anonymous iterator blocks do, and have a much more compelling end user benefit.

In short, the benefits were worth the cost to implement, unlike iterator blocks. The costs were likely rather comparable.

Servy
  • 202,030
  • 26
  • 332
  • 449
  • It starts becoming a nuisance when you have to refactor your iterator blocks into new classes, because you need to capture some local variables in them. And even then it is not a big nuisance - I just wondered why the compiler does this for async lambdas, but not for iterators in lambdas, when the mechanism is the same. – jakobbotsch Dec 31 '13 at 20:26
  • 1
    [why doesn't C# implement feature X?](http://blogs.msdn.com/b/ericlippert/archive/2009/06/22/why-doesn-t-c-implement-top-level-methods.aspx), "The answer is always the same: because no one ever designed, specified, implemented, tested, documented and shipped that feature. All six of those things are necessary to make a feature happen. All of them cost huge amounts of time, effort and money. Features are not cheap, and we try very hard to make sure that we are only shipping those features which give the best possible benefits to our users given our constrained time, effort and money budgets." – Erik Philips Dec 31 '13 at 21:02
  • @Janiels Times at which you would want to close over a local in for an iterator block is generally a case where you can accept a delegate, and then have that delegate close over the variable. – Servy Dec 31 '13 at 22:07
  • @Erik Philips Yes, but this question was not only of the form "Why doesn't C# implement feature X?", it was of the form "Why doesn't C# implement feature X, when it looks so much like feature Y which is already implemented?". As Eric Lippert pointed out, it is because under the hood, they aren't as similar as I anticipated, in which case your link makes sense; and I completely agree with, and accept that. – jakobbotsch Jan 01 '14 at 14:29
1

Look at this code (it does not work, just an example):

Task<IEnumerable<int>> resultTask = new Task<IEnumerable<int>>(() =>
{
    for (int i = 0; i < 10; ++i)
    {
        yield return i;
    }
});

Don't you find it some kind of unstructured?

Assuming whole range of lambdas usages, it would be so hard and not worth it to handle yield "laziness" properly.

However, there are great approaches to yield return from parallel tasks.

But let's have a look on following thing. Defining a method with yield return:

static IEnumerable<int> GetIntegers()
{
    for (int i = 0; i < 10; ++i)
    {
        yield return i;
    }
}

And putting it in lambda will work:

Task<IEnumerable<int>> resultTask = new Task<IEnumerable<int>>(() =>
{
    return GetIntegers();
});

What is the way this code will behave in? Is it going to lose real yield advantages?

Community
  • 1
  • 1
Ilya Tereschuk
  • 1,204
  • 1
  • 9
  • 21