Performance of IEnumerable

Question

I have this code

List<int> items = new List<int>() { 1, 2, 3, 4, 5, 6, 7, 8, 9, 0 };
//calculation is not being executed until "even" is used
IEnumerable<int> even = items.Where(x => x % 2 == 0);   

DoStuff1(even);
DoStuff2(even);
DoStuff3(even);

I've read this answer https://stackoverflow.com/a/3628705/6887468 and it says

When you use IEnumerable, you give the compiler a chance to defer work until later, possibly optimizing along the way. If you use ToList() you force the compiler to reify the results right away.

Now is this calculation (in my example x % 2 == 0) being executed for each call of DoStuff() or is this somehow hold in the Memory?

Suppose you add an item after after your where.`IEnumerable even = items.Where(x => x % 2 == 0); `beneath it items.Add(10). it will be still counted on your `DoStuff` as even is still not evaluated. — Eldho, Jul 26 '18 at 09:01
If `DoStuffx` iterates over the `even` parameter then `x % 2 == 0` will execute **inside** each of the `DoStuffx` methods. If you used `ToList` **before** calling the methods then the `x % 2 == 0` will occur only once (prior to any of the `DoStuffx` calls) per element in the original `List`. — mjwills, Jul 26 '18 at 09:14
See https://stackoverflow.com/questions/16946207/does-foreach-cause-repeated-linq-execution. https://stackoverflow.com/q/5050326/613130, https://stackoverflow.com/questions/8240844/handling-warning-for-possible-multiple-enumeration-of-ienumerable — xanatos, Jul 26 '18 at 09:21

bommelding · Accepted Answer · 2018-07-26T09:31:59.250

3

executed for each call of DoStuff() or is this somehow hold in the Memory?

It is not by itself held in memory so the following is sometimes a worthwhile optimization:

IEnumerable<int> even = items.Where(x => x % 2 == 0);   
even = even.ToList();  // now it is held in memory
DoStuff1(even);
DoStuff2(even);
DoStuff3(even);

Whether you want the ToList() depends on what DoStuff() does, how big the source list is vs the filtered list, how expensive the enumeration, etc.

Note that when items was a little longer and all the DoStuff methods had a main loop like foreach(var item in items.Take(3)) then the ToList() in the main method would be an expensive de-optimization.

edited Jul 26 '18 at 09:31

answered Jul 26 '18 at 09:05

bommelding

2,969
9
14

Carbine just deleted it – Toshi Jul 26 '18 at 09:13
3

@Toshi that answer is wrong. .Net doesn't explicitly store that in memory or cache. It will execute multiple times your .Where statement. And there is a Resharper/VS warning about multiple enumeration of IEnumerable – dlxeon Jul 26 '18 at 09:13
1

@Toshi sorry I was wrong about my assumption. Corrected it – Carbine Jul 26 '18 at 09:23

score 1 · Answer 2 · answered Jul 26 '18 at 09:18

Clarification:

In your line:

IEnumerable<int> even = items.Where(x => x % 2 == 0);

the expression is not evaluated, so the iteration does not actually happen. This is what referred to 'deferred evaluation'. Note: This is conceptionally independent of the interface implementation, so regardless it is a List<> or [] or etc. (of course it is possible to do an implementation against this main concept, but this is out of scope of the question)

The execution start happen when you start iterate on even, for example with foreachor GetEnumerator() then Next() or FirstOrDefault or ToList() or ToArray() etc.

Although we do not see your DoStuffX() code likely you do something similar.

Answer

What ar you asking is mainly independent of deferred evaluation described above: It is about caching. If you do not cache the result the iteration happens multiple times. The simplest mode to evaluate (iterate) and cache is execute a ToList()

var cached = even.ToList();

(Please note not all IEnumerable implementation allows multiple iterasions, but List<> does.

score 0 · Answer 3 · answered Jul 26 '18 at 09:00

How CAN he hold it in memory? Not püossible. Must execute.

Deferred does not mean caching. It means that the line

IEnumerable even = items.Where(x => x % 2 == 0);

MAY NOT execute ANYTHING. No joke - it may be deferred.

And then in the line

DoStuff1(even);

it MAY execute.

When items is not an in memory list but a database, possibly a query that is cmplex, people come and say "oh, my foreach spends 1 minute before it gets the first line" - THAT is what deffered means. It will in this case just send the SQL and execute it when the first item is requested.

Performance of IEnumerable

3 Answers3