Break large array object to smaller chunks

Question

I have an Array, which can vary from 1 - 600K records. Therefore, I need to find a way to break this bigger array into smaller chunks and perform some operation of the smaller chunks.

How can I do this?

My solution as follows: My problem is that I am not sure, how many elements the array will contain so I am not able to divide it by 10, to determine the chunk size.

for(const i = 0 ; i < largeArray.length; largeArray.length/10) {
   var p1 = largeArray.slice(i,whatisthechunksize);
}

Possible duplicate of [How to split a long array into smaller arrays, with JavaScript](https://stackoverflow.com/questions/7273668/how-to-split-a-long-array-into-smaller-arrays-with-javascript) — Prashant Pimpale, May 20 '19 at 12:11
yes, I did! and I think it's relevant/duplicate! but need small modification! also I rad the line before `How can I do this?` — Prashant Pimpale, May 20 '19 at 12:20
And what is the problem with If the array items are 11 and used above code? — Prashant Pimpale, May 20 '19 at 12:24

JuanDM · Answer 1 · 2019-05-20T12:18:35.227

You can use this function, it will return an array of chunks considering the total elements of the original array.

function splitArray(array, chunk) {
  const chunkSize = chunk || 10;

  const chunkedArray = array.reduce((acc, item) => {
    // delete the last item from acumulator
    // (it is made until the group get all the chunk items)
    let group = acc.pop();

    // validate if the group has the size defined
    if (group.length === chunkSize) {
      acc.push(group);
      group = [];
    }
    // Insert in the chunk group
    group.push(item);

    // push the group to the reduce accumulator
    acc.push(group);
    return acc;
  }, [[]]); // [[]] is used to initialize the accumulator with an empty array

  return chunkedArray;
}

score 0 · Answer 2 · answered May 20 '19 at 12:19

0

Something along these lines?

import { from } from 'rxjs';
import { skip, take } from 'rxjs/operators';

const source = from(array);

const p1 = source.pipe(
  skip(10 * i),
  take(10)
);

answered May 20 '19 at 12:19

Mike Jerred

9,551
5
22
42

score 0 · Answer 3 · answered May 20 '19 at 12:25

Use Array.prototype.reduce. You can also extend that prototype to split by the desired number :

Array.prototype.splitBy = function(size) {
  return this.reduce((p, n) => {
    if (p[p.length - 1].length < size) p[p.length - 1].push(n);
    else p.push([n]);
    return p;
  }, [[]]);
}

const data = new Array(500).fill(0).map((v, i) => i);

const splitted = data.splitBy(10);

console.log(splitted);
console.log('Size of splitted array = ', splitted.length);

score 0 · Answer 4 · answered May 20 '19 at 12:33

I would go with this! Simple and Clean!

Copied from here

var size = 3; var arrayOfArrays = [1,2,3,4,5,6,7,8,9,10];

for (var i=0; i < arrayOfArrays.length; i+=size ) {
     var chunckedArray = arrayOfArrays.slice(i,i+size);
     console.log(chunckedArray); 

     for(var j = 0; j < chunckedArray.length; j++)
     {
       console.log(chunckedArray[j]); // Do operations
     }   
}

Break large array object to smaller chunks

4 Answers4