deferred execution or not

Question

Are below comments correct about DEFERRED EXECUTION?

1. var x = dc.myTables.Select(r=>r);//yes
2. var x = dc.myTables.Where(..).Select(r=>new {..});//yes
3. var x = dc.myTables.Where(..).Select(r=>new MyCustomClass {..});//no

In other words, I always thought projecting custom class objects will always cause eager execution. But I couldn't find references supporting/denying it (though I am seeing results contradicting it, hence the post)

You need to expand upon this: _though I am seeing results contradicting it_ — Austin Salonen, Mar 15 '13 at 15:54
Could this behaviour depend on the nature of the `MyCustomClass` constructor? — paul, Mar 15 '13 at 15:55
@AustinSalonen - I meant, 'I am seeing results contradicting my assumption'. I try to use x in 3. later, but getting exceptions saying dc is already disposed — Brian, Mar 15 '13 at 16:02
Post a code snippet where that's true of #3 but not true of #1 and #2. — Austin Salonen, Mar 15 '13 at 16:06
@AustinSalonen - I meant I am seeing results contradicting my assumption that 'projecting custom class objects will cause eager execution'. Guess I wasn't clear. — Brian, Mar 15 '13 at 16:30
@AustinSalonen - Suppose I had say stmt1 - IEnumerable<> x, stmnt2. using(dc){ x= project customclassobjects} stmnt3. access x stmnt3 throwing exception because dc is disposed already, so it implies not eager execution. — Brian, Mar 15 '13 at 16:37
All you've proven is that `using` works as expected. To show _not eager_, your mapping (Select) needs to do something you can observe (say write to the console) and assert that nothing was "done". For example, `r => { Console.WriteLine("qqqxxx"); return new MyCustomClass(...); }` (though linq-to-sql may not like this...) — Austin Salonen, Mar 15 '13 at 17:39
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/26262/discussion-between-brian-and-austin-salonen) — Brian, Mar 15 '13 at 18:47
@AustinSalonen - I guess I expected (if eager execution,), x would persist still, outside the using{} — Brian, Mar 15 '13 at 18:51
Nope. The using translates to `try { x = ...; }finally {x.Dispose();}`. Given how you defined it, `x` will be in scope outside of the using but with it set in the `using` it will be disposed when you use it later. — Austin Salonen, Mar 15 '13 at 18:59

score 7 · Accepted Answer · answered Mar 15 '13 at 15:53

Every statement in your question is an example of deferred execution. The contents of the Select and Where statement have no effect on whether or not the resulting value is deferred executed or not. The Select + Where statements themselves dictate that.

As a counter example consider the Sum method. This is always eagerly executed irrespective of what the input is.

var sum = dc.myTables.Sum(...);  // Always eager

Austin Salonen · Answer 2 · 2013-03-15T19:15:23.080

3

To prove your point, your test should look like this:

var tracer = string.Empty;
Func<inType, outType> map = r => {
       tracer = "set";
       return new outType(...);
    } 

var x = dc.myTables.Where(..).Select(map);

// this confirms x was never enumerated as tracer would be "set".
Assert.AreEqual(string.Empty, tracer);
// confirm that it would have enumerated if it could
CollectionAssert.IsNotEmpty(x);

edited Mar 15 '13 at 19:15

answered Mar 15 '13 at 19:08

Austin Salonen

49,173
15
109
139

1

You can even have the iterator throw an exception when first iterated, making it very clear exactly when the first item is actually requested from it. – Servy Mar 15 '13 at 19:24

score 2 · Answer 3 · answered Mar 15 '13 at 15:53

2

It has been my observation that the only way to force execution right away is to force iteration of the collection. I do this by calling .ToArray() on my LINQ.

answered Mar 15 '13 at 15:53

Malcolm O'Hare

4,879
3
33
53

1

ToArray? ToList is better. – Matías Fidemraizer Mar 15 '13 at 15:55
2

@MatíasFidemraizer That would depend on what you intend on doing with the results. The implementations of the two functions are pretty much the same so I doubt there is any performance hit. In my case I cannot think of examples where I'd want to use LINQ to create a list that I'd want to add items to afterwards. – Malcolm O'Hare Mar 15 '13 at 15:58
@MalcolmO'Hare No performance issues. Arrays _as is_ have limited use cases, while lists and collections in general terms are the recommended objects to store objects in OOP and specially in .NET/C#. – Matías Fidemraizer Mar 15 '13 at 16:01
1

@MatíasFidemraizer: _general terms are the recommended objects to store objects in OOP and specially in .NET/C#_ -- [citation needed] – Austin Salonen Mar 15 '13 at 16:04
@AustinSalonen I'm not going to elaborate on this in a comment. Just _google_ this topic and you'll find a lot of info. In addition, StackOverflow has many Q&A about this. Just do a search :D – Matías Fidemraizer Mar 15 '13 at 16:06

score 2 · Answer 4 · edited May 23 '17 at 11:50

2

Generally methods that return a sequence use deferred execution:

IEnumerable<X> ---> Select ---> IEnumerable<Y>

and methods that return a single object doesn't:

IEnumerable<X> ---> First ---> Y

So, methods like Where, Select, Take, Skip, GroupBy and OrderBy use deferred execution because they can, while methods like First, Single, ToList and ToArray doesn't because they can't.

from here

edited May 23 '17 at 11:50

Community

1
1

answered Mar 15 '13 at 15:56

Scott Selby

9,420
12
57
96

It's a good deduction, while it's not absolutely necessary that you return a sequence in order to do _deferred execution_. But yes, in .NET BCL this is true! – Matías Fidemraizer Mar 15 '13 at 16:05
Well, having deferred execution isn't just a boolean yes/no. It's more complex than that. For example, `Where` doesn't invoke the predicate on each item until the last possible moment, right before that item is requested by the iterator, whereas `GroupBy` or `OrderBy` on the other hand will need to eagerly process the entire result set as soon as the first item is requested, so they can only defer execution from the time the query is created until the first iteration. Then there are cases like `Intersect` that eagerly evaluate only one sequence when the first item is requested. – Servy Mar 15 '13 at 19:23

Matías Fidemraizer · Answer 5 · 2013-03-16T08:14:36.750

2

.Select(...) is always deferred.

When you're working with IQueryable<T>, this and other deferred execution methods build up an expression tree and this isn't ever compiled into an actual executable expression until it's iterated. That is, you need to:

Do a for-each on the projected enumerable.
Call a method that internally enumerates the enumerable (i.e. .Any(...), .Count(...), .ToList(...), ...).

edited Mar 16 '13 at 08:14

answered Mar 15 '13 at 15:59

Matías Fidemraizer

63,804
18
124
206

`isn't ever compiled into an actual executable expression until it's iterated. That is, you need to:` that only applies to the `IQueryable` overload. For the `IEnumerable` overload it's compiled into executable code at compile time, not at runtime. It's a delegate that simply isn't called until iteration. – Servy Mar 15 '13 at 19:20
@Servy Are you meaning that Select on IEnumerable isn't part of an expression tree? – Matías Fidemraizer Mar 15 '13 at 20:08
That is correct. The IEnumerable select takes a `Func>`. As similar as they look, they are very different. – Servy Mar 15 '13 at 20:14
@Servy But, after checking the tags on the question, I feel that the answer is still correct: it's for linq-to-sql (IQueryable). By the way, I don't know why, I thought `Select(...)` was expression in any situation (it could be as simple as looking at `IEnumerable` `Select(...)` overloads... – Matías Fidemraizer Mar 16 '13 at 08:10

deferred execution or not

5 Answers5