1

I have to generate an 80x250,000 CSV through a browser. Handling this much data is somewhat slow but manageable in arrays, but I have a problem when I try to generate a CSV string to turn into a Blob.

Solutions such as this: https://stackoverflow.com/a/25975345 involve creating a large string of the CSV contents. I cannot create such a string without crashing the browser. How can I generate a file this large? Below is the code I'm currently using to benchmark this:

var fileData = "contents:text/csv;base64,";
for(var i = 0, temp; i < 150000; i++){
    for(var j = 0; j < 80; j++){
        fileData += Math.random() + ",";
    }
    fileData += "\r\n";
}
var exportLink = document.createElement('a');
var csvData = new Blob([window.btoa(fileData)], { type: "text/csv" }); 
var csvUrl = URL.createObjectURL(csvData);
exportLink.href =  csvUrl;
Community
  • 1
  • 1
jokul
  • 1,269
  • 16
  • 37
  • 1
    Why can't you create the file on the server-side and return it using AJAX? – BenM Dec 12 '16 at 22:28
  • Why do you need to generate it in the browser? (instead of generating it server-side) – abl Dec 12 '16 at 22:28
  • 1
    We don't have the ability to create any server-side interfaces. The entire back end is part of a prepackaged product. – jokul Dec 12 '16 at 22:30
  • @jokulmorder What? If you have access to a web server, you can create this on the backend. Heavy lifting like this was never designed to be done in the browser. – BenM Dec 12 '16 at 22:31
  • 1
    We don't have access to the web server. If it can't be done, it can't be done. – jokul Dec 12 '16 at 22:32
  • 2
    Then you may consider setting up another server (which you do have access to) that gets the data from the one you cannot touch and prepares the file for download. – abl Dec 12 '16 at 22:34
  • I didn't say *the* web server, I said *a* web server. Since all the file appears to do is create a matrix of random numbers, this could be achieved on any web server. – BenM Dec 12 '16 at 22:34
  • We can't do that either. That decision is not ours to make. Trust me, I would prefer that solution too and I've used it in the past to get around memory limitations in the browser. Our next move will be to dynamically generate several CSVs of maybe 80x50K instead. – jokul Dec 12 '16 at 22:37
  • Also, the random numbers were solely to benchmark whether or not this is possible since I had serious doubts about data of this magnitude being handled by the client. There is actual data. – jokul Dec 12 '16 at 22:39
  • http://stackoverflow.com/questions/27081858 – abl Dec 12 '16 at 23:07

0 Answers0