2

I have the following:

IEnumerable<string> lines // passed into method

foreach (var data in lines.ParseAsEnumberable(Delimiter, StringQualifer))
{
...
}

This is fine for this particular foreach loop. But I want to go through the lines again, but get a

"possible multiple enumeration"

warning if I use a similar loop again, and sure enough, the second time there are no enumeration results.

How do I go through the lines a second time?

Edited to add: I have been informed by my boss that we should not use ToList, as we have files with many gigabytes of data. What can I use instead?

Scott
  • 2,456
  • 3
  • 32
  • 54
  • 4
    Materialize the `IEnumerable`, turn it into array, list etc. collection: `var list = lines.ParseAsEnumberable(Delimiter, StringQualifer).ToList()` and then loop over `list` – Dmitry Bychenko Mar 14 '23 at 11:12
  • My boss said we should not use ToList, as we have files with many gigabytes of data. What should I use instead? – Scott Mar 15 '23 at 00:38
  • I see; since the main problem is potential inconsistency you can try doing all the tasks within single `foreach` (which is preferred way out: enumerating *multigigabyte* files *several times* is not a good idea) or lock the file (https://stackoverflow.com/questions/5522232/how-to-lock-a-file-with-c) to prevent anyone to modified and then loop as many times as you have to do and *supress* with `pragma`: `#pragma warning disable CA1851...#pragma warning restore CA1851` (https://learn.microsoft.com/en-us/dotnet/fundamentals/code-analysis/quality-rules/ca1851) – Dmitry Bychenko Mar 15 '23 at 08:51

1 Answers1

6

The problem with IEnumerable<T> that is in general case it is not consistent, e.g. when we read from outer source (like file, RDBMS table, COM port etc.)

IEnumerable<string> lines = File
  .ReadLines(@"c:\...")
  ...

we can have different data when we enumerate lines second time (file, table can be changed). To avoid this possible inconsistency you can materialize the enumeration:

// Materialization can be required, but 
// let's avoid overhead if lines is already a collection.
// From now on source is a collection; it doesn't change between loops
var source = lines as IReadOnlyCollection<string> ?? lines.ToList();

...

// 1st loop
foreach(var data in source.ParseAsEnumberable(Delimiter, StringQualifer)) {
  ...
}

...

// 2nd loop over the very same collection
foreach(var item in source) {
  ...
}

Edit: if you absolutely sure that you have to loop over enumeration several times and you are not going to have problems with inconsistency, you can suppress the warning with a help of pragma:

//TODO: Provide justification why it is safe to break the rule here
// E.g. "The file is locked and can't be modified"
#pragma warning disable CA1851

foreach (var data in lines.ParseAsEnumberable(Delimiter, StringQualifer)) {
  ...
}

...

foreach (var item in lines) {
  ...
}

// Don't forget to restore: 
// we've adjusted just one exception from the general rule 
#pragma warning restore CA1851
Dmitry Bychenko
  • 180,369
  • 20
  • 160
  • 215