Using Javascript FileReader with huge files

Question

I have a problem using the Javascript FileRead trying to read huge files.

For example, I have a text file of 200mb and everytime I read this file the code stops working.

Its possible to read the text file, but for example ONLY the first 10 lines or stop reading after 10mb?

This is my code:

var file = form.getEl().down('input[type=file]').dom.files[0];
var reader = new FileReader();

reader.onload = (function(theFile) {
                return function(e) {
                    data = e.target.result;
                    form.displayedData=data;
                };
            })(file);

reader.readAsText(file);

The e.target.result always has the whole data of the file.

What can I do here?

Thx

score 13 · Accepted Answer · answered Jul 04 '13 at 14:06

13

This will only read the first 10 mb:

var file = form.getEl().down('input[type=file]').dom.files[0];
var reader = new FileReader();

reader.onload = function(e) {
    var data = e.target.result;
    form.displayedData = data;
};

reader.readAsText(file.slice(0, 10 * 1024 * 1024));

answered Jul 04 '13 at 14:06

Esailija

138,174
23
272
326

2

Yes this read first 10mb but file is still in memory, i think the original question was about reading huge files and not loading whole file into memory. Am I not right? – Yetti Feb 08 '15 at 10:22
Yes @Yetti my problem is same. if we try open 4GB file, file.slice will try load entire file. – Fatih Apr 02 '16 at 06:19
2

@Yetti no it only loads 10mb into memory. – Esailija Apr 07 '16 at 10:57
1

@Fatih `Blob.slice` (same as `File.slice`) _shouldn't_ load the entire file into memory, it's basically true file streaming if you only read by slices (that said though, the actual underlying behavior is browser and system specific, e.g. it's possible that a 200MB file will be loaded into memory on a 64GB RAM system, but not a 20GB file). Related: https://stackoverflow.com/questions/24833487/what-is-html5-file-slice-method-actually-doing and https://chromium.googlesource.com/chromium/src/+/master/storage/browser/blob/README.md – John Weisz Mar 27 '19 at 10:36
Thanks @JohnWeisz I'll try. – Fatih Mar 28 '19 at 19:29

Using Javascript FileReader with huge files

1 Answers1

Linked