4

I’d like to use javascript to create and save a large .csv file client-side. This has been asked and answered before (see 1, 2, 3, or 4, for example), and this comes down to two approaches (among others):

  1. Use FileSaver.js to implement w3c’s saveAs():

    var lines = ["line 1,cell 11,cell 12\n","line 2,cell 21,cell 22\n"];
var type = "text/csv;charset=utf-8";
var blob = new Blob(lines,{"type":type});
saveAs(blob,"download.csv");

  2. Use a[download] and a data uri:

    var lines = ["line 1,cell 11,cell 12\n","line 2,cell 21,cell 22\n"];
var type = "text/csv;charset=utf-8";
var downloader = $('<a download href="data:'+type+','+escape(lines.join(''))+'"></a>')
    .appendTo("body")
downloader[0].click();
downloader.remove();

My problem is that my file can be gigantic: 18 million lines and 2.5 Gb (and I’d like to be able to handle more). Creating lines uses too much memory and crashes the site. But there’s no reason to store lines in the browser’s memory just to save it to the hard drive. Is there a way to create a progressive download using javascript (in other words, start the download but keep appending lines as I calculate them)? Or would my only option be to download the file as separate chunks that the user must then join together?

Community
  • 1
  • 1
Teepeemm
  • 4,331
  • 5
  • 35
  • 58
  • While the question is well-written and properly-structured I think that the possible answers will be too opinion based. – Jonast92 Feb 23 '15 at 17:47
  • 3
    @Jonast92 I would agree with this being too opinion based if my question were "which of my approaches is better". But my question is "is what I want to do at all possible", which doesn't seem to be opinion based (IMO). – Teepeemm Feb 23 '15 at 18:03
  • I see. Perhaps the best solution is simply to _try_ both and see which is more suitable by taking a look at some benchmarks, then you'll know the answer, at least for your scenario, and you could share it with the world! :) --Although I do understand that you'd like to spare some time by waiting for an answer but you should probably not count on it. – Jonast92 Feb 23 '15 at 18:10
  • 2
    @Jonast92 I’m not sure you’re understanding me. I've tried both approaches. Google Chrome reports that the tab’s memory goes to 1.5 Gb, the cpu to 135-140 (whatever that means), and the tab crashes, presumably because `lines` is too big. Hence my desire to save the file to my hard drive without creating the entire object in javascript's memory. Something like `blob=new Blob(); saveAs(blob); blob.append(lines);` in that order. – Teepeemm Feb 23 '15 at 18:49

1 Answers1

0

If you are using HTML5, you can try using the FileSytem API. See here for information on the subject, and here for a more practical example on how to create and access files fron the client-side.

imelgrat
  • 2,407
  • 1
  • 12
  • 17
  • I am indeed using html5. Thanks for the links, and [html5rocks](http://www.html5rocks.com/en/tutorials/file/filesystem/) seems to provide even more examples. Unfortunately, though, it appears that [FileSystem has been discontinued](http://caniuse.com/#search=filesystem), and I expect that support for this approach will only decline in the future. – Teepeemm Feb 23 '15 at 20:01
  • The FileSystem API doesn't have full support, only Chrome, Opera and some mobile browsers support it (basically, Webkit-based browsers). Since yours is a rather uncommon app, I thought this suggestion might have worked for you, and that browser support mightcome second on your list, after being able to generate these large files on the client. – imelgrat Feb 23 '15 at 21:56
  • The FileSystem has been discontinued but it's not likely that they are going to remove it. Too many web apps depends on it even the chrome extension use it. – Endless Jun 13 '16 at 20:27