-1

I have an extension method that will split datatables in to size mentioned.Suppose if i say 1000 and if i have 10000 items ,it will create 10 datatables of 1000 items.I items are more then it will create lot of datatables.But what i want is what ever items we have,i want 5 datatables only.How to achieve it.The code i have is

var tables = dt.AsEnumerable().ToChunks(10000)
                          .Select(rows => rows.CopyToDataTable())
    public static IEnumerable<IEnumerable<T>> ToChunks<T>(this IEnumerable<T> enumerable,
                                                          int chunkSize)
    {
        int itemsReturned = 0;
        var list = enumerable.ToList(); // Prevent multiple execution of IEnumerable.
        int count = list.Count;
        while (itemsReturned < count)
        {
            int currentChunkSize = Math.Min(chunkSize, count - itemsReturned);
            yield return list.GetRange(itemsReturned, currentChunkSize);
            itemsReturned += currentChunkSize;
        }
    }

EDIT

I have 50000 records and i want to process it in TPL tasks.In this code it will create 50 datatables and 50 tasks which is not good.So i want 5 datatables each contains 10000 records and 5 tasks.This is not fixed , i mean if it is 60000 then 5 tasks and 5 tables with 12000 records each

peter
  • 8,158
  • 21
  • 66
  • 119
  • A DataTable can hold 16,777,216 rows [source](https://stackoverflow.com/questions/11123074/what-is-the-maximum-size-of-a-datatable-i-can-load-into-memory-in-net), it feels like a poor design decision to split it apart. Perhaps you can share why you want it split apart and someone might be able to give you a hand. – Michael Woolsey Nov 07 '17 at 20:47
  • I have 50000 records and i want to process it in TPL tasks.In this code it will create 50 datatables and 50 tasks which is not good.So i want 5 datatables each contains 10000 records and 5 tasks.This is not fixed , i mean if it is 60000 then 5 tasks and 5 tables with 12000 records each – peter Nov 08 '17 at 08:39

1 Answers1

0

With the very nice implementation here you could do:

var array = sequence.ToArray();
var numberOfSlices = 5;
var numberOfElementsInSlice = Math.Ceil(array.Length / (double)numberOfSlices);
var sliceRanges = Enumerable.Repeat(numberOfElementsInSlice, numberOfSlices).ToArray();

var slices = array.Slice(sliceRanges);
nvoigt
  • 75,013
  • 26
  • 93
  • 142