How to chunk string using javascript

Question

I have a string which is more than 32kb it needs to be chunked, with every chunk having a size limit of 32kb.is it possible ? using JavaScript , I can only find codes like cutting the string or splitting the string in which, I think is not related to my task

stringChop = function(str, size){
if (str == null)
return [];
str = String(str);
return size > 0 ? str.match(new RegExp('.{1,' + size + '}', 'g')) : [str];
}

I also have code the check the bytes

const byteSize = str => new Blob([str]).size;

const result = byteSize("sample")

If you do that with Node.js, simply read your input file line by line, store those dynamic chunks into buffer string and once the size of the buffer reaches the limit, push it into separate file. — Yevhen Horbunkov, Dec 07 '20 at 11:25
@YevgenGorbunkov so using javascript will not do the tricks ? im getting the string from image base64 encode using java then send it to javascript — Drian Drian, Dec 07 '20 at 11:29
Here you will find some answers, https://stackoverflow.com/questions/32898082/splitting-a-file-into-chunks-with-javascript — lissettdm, Dec 07 '20 at 12:20
And here, https://stackoverflow.com/questions/57068850/how-to-split-a-string-into-chunks-of-a-particular-byte-size — lissettdm, Dec 07 '20 at 12:22

Benjamin Gruenbaum · Answer 1 · 2020-12-07T11:54:55.577

You really don't want to "spend time" splitting large strings in Node.

If you have to use vanilla

This is entirely possible with JavaScript (and you're pretty close). Though this is more elegant without regular expressions and with generators:

function* chunk(str, size = 3) {
  for(let i = 0; i < str.length; i+= size ) yield str.slice(i, i + size);
}
[...chunk('hello world')]; // ["hel", "lo ", "wor", "ld"];

If you can use Node.js

I'd read the file you want to split with a createReadStream and then write it to different files when it reaches the limit. This is much more effective since you don't create many small strings or keep all the data in memory:

(async () => {
  let currentFileIndex = 0, currentBytes = 0;
  let currentFile = fs.createWriteStream(`${currentFileIndex}.csv`);
  for await(const chunk of fs.createReadStream('input.csv') {
    currentBytes += chunk.length;
    if (currentBytes > 32000) { // or whatever limit you want
      currentFile.end(); // probably wait for the allback here
      currentBytes = 0;
      currentFile = fs.createWriteStream(`${++currentFileIndex}.csv`)
    }
    await util.promisify(cb => currentFile.write(chunk, cb)();
  }
})();

How to chunk string using javascript

1 Answers1

If you have to use vanilla

If you can use Node.js