Is there a way to Memorize or Materialize an IEnumerable?

Question

When given an d you could be dealing with a fixed sequence like a list or array, an AST that will enumerate some external datasource, or even an AST on some existing collection. Is there a way to safely "materialize" the enumerable so that enumeration operations like foreach, count, etc. don't execute the AST each time?

I've often used .ToArray() to create this represenation but if the underlying storage is already a list or other fixed sequence, that seems like wasted copying. It would be nice if i could do

var enumerable = someEnumerable.Materialize();

if(enumberable.Any() {
  foreach(var item in enumerable) {
    ...
  }
} else {
  ...
}

Without having to worry that .Any() and foreach try to enumerate the sequence twice and without it unccessarily copying the enumerable.

Answer 1

Original answer:

Same as Thomas's answer, just a bit better according to me:

public static ICollection<T> Materialize<T>(this IEnumerable<T> source)
{
    // Null check...
    return source as ICollection<T> ?? source.ToList();
}

Please note that this tend to return the existing collection itself if its a valid collection type, or produces a new collection otherwise. While the two are subtly different, I don't think it could be an issue.

---

Edit:

Today this is a better solution:

public static IReadOnlyCollection<T> Materialize<T>(this IEnumerable<T> source)
{
    // Null check...
    switch (source)
    {
        case IReadOnlyCollection<T> readOnlyCollection:
            return readOnlyCollection;

        case ICollection<T> collection:
            return new ReadOnlyCollectionAdapter<T>(collection);

        default:
            return source.ToList();
    }
}

public class ReadOnlyCollectionAdapter<T> : IReadOnlyCollection<T>
{
    readonly ICollection<T> m_source;

    public ReadOnlyCollectionAdapter(ICollection<T> source) => m_source = source;

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    public int Count => m_source.Count;

    public IEnumerator<T> GetEnumerator() => m_source.GetEnumerator();
}

Mind you the above solution misses a certain covariant case where the collection type implements ICollection<T> but not IReadOnlyCollection<T>. For e.g. consider you have a collection like below:

class Collection<T> : ICollection<T>
{
}

// and then
IEnumerable<object> items = new Collection<Random>();

The above compiles since IEnumerable<T> is covariant.

// later at some point if you do
IReadOnlyCollection<object> materialized = items.Materialize();

The above code creates a new List<Random> (O(N)), even though we passed an already materialized collection. The reason is ICollection<T> is not a covariant interface (it can't be), hence our cast from Collection<Random> to ICollection<object> fails, so the default: case in the switch is executed.

I believe it is an extremely rare scenario for a collection type to implement ICollection<T> but not IReadOnlyCollection<T>. I would just ignore that case. Scanning BCL libraries I could find only very few and that too hardly heard of. If at all you need to cover that case as well, you could use some reflection. Something like:

public static IReadOnlyCollection<T> Materialize<T>(this IEnumerable<T> source)
{
    // Null check...

    if (source is IReadOnlyCollection<T> readOnlyCollection)
        return readOnlyCollection;

    if (source is ICollection<T> collection)
        return new ReadOnlyCollectionAdapter<T>(collection);
    
    // Use your type checking logic here.
    if (source.GetType() (is some kind of typeof(ICollection<>))
        return new EnumerableAdapter<T>(source);

    return source.ToList();
}

public class EnumerableAdapter<T> : IReadOnlyCollection<T>
{
    readonly IEnumerable<T> m_source;

    public EnumerableAdapter(IEnumerable<T> source) => m_source = source;

    IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();

    public int Count => ((dynamic)m_source).Count;

    public IEnumerator<T> GetEnumerator() => m_source.GetEnumerator();
}

Answer 2

Easy enough:

public static IList<TSource> Materialize<TSource>(this IEnumerable<TSource> source)
{
    if (source is IList<TSource>)
    {
        // Already a list, use it as is
        return (IList<TSource>)source;
    }
    else
    {
        // Not a list, materialize it to a list
        return source.ToList();
    }
}

Answer 3

Check out this blog post I wrote a couple of years ago: http://www.fallingcanbedeadly.com/posts/crazy-extention-methods-tolazylist

In it, I define a method called ToLazyList that effectively does what you're looking for.

As written, it will eventually make a full copy of the input sequence, although you could tweak it so that instances of IList don't get wrapped in a LazyList, which would prevent this from happening (this action, however, would carry with it the assumption that any IList you get is already effectively memoized).