How to process IEnumerable in batches?

Question

A lot of time my lists are too big tor process and i have to break them up into batches to process.

Is there a way to encapsulate this in a method or an extension. I seem to have to write this batching logic everywhere.

const int TAKE = 100;
int skip = 0;
while (skip < contacts.Count)
{
    var batch = contacts.Skip(skip).Take(TAKE).ToList();
    DoSomething(batch);
    skip += TAKE;
}

I would like to do something like -

Batch(contacts, DoSomething);

Or anything similar so i do not have to write this batching logic again and again.

Answer 1

Using the batch solution from this thread it seems trivial:

const int batchSize = 100;

foreach (var batch in contacts.Batch(batchSize))
{
    DoSomething(batch);
}

If you want to also wrap it up:

public static void ProcessInBatches<TSource>(
              this IEnumerable<TSource> source,
              int batchSize,
              Action<IEnumerable<TSource>> action)
{
    foreach (var batch in source.Batch(batchSize))
    {
        action(batch);
    }
}

So, your code can be transformed into:

const int batchSize = 100;

contacts.ProcessInBatches(batchSize, DoSomething);