chrome extension crashes when i try to upload a big file

Question

The extension crashes when it executes such code:

/*this object is created in content script and passed t background script*/
var myurl = URL.createObjectURL(document.getElementById('myfile').files[0]);

/*code block from background script, it work good if file size is < 50MB, if bigger then extension will crash*/
var x = new XMLHttpRequest();
x.onload = function() {
       var uploadfile = new Uint8Array(x.response);


        var somearray1 = [...];
        var somearray2 = [...];

        var size = somearray1.length + uploadfile.length + somearray2.length;

        var u8array = new Uint8Array(size);
        var i = 0;

         for (i = 0; i < somearray1.length; i++) 
         u8array[i] = somearray1.charCodeAt(i) & 0xff;


        for (var j = 0; j < uploadfile.length; i++, j++) 
        u8array[i] = ufile[j];

        for (i = 0; i < somearray2.length; i++) 
         u8array[i] = somearray2.charCodeAt(i) & 0xff;


        var req = new XMLHttpRequest();

        req.open("POST", Url);
        req.setRequestHeader("Content-Type",  'multipart/form-data; boundary=--_BOUNDARY_');
        req.send(u8array);

  };
x.open('GET', myurl); 
x.responseType = 'arraybuffer';
x.send();

I want to upload a file of 200MB size, and it crashes the extension. Please help me understand with some sample code how to upload it correctly if it is wrong the way i'm doing now.

I think yes, the browser only show a notice that extension crashed, and disabled the extension and says to click the notice to enable the extension if i want. But if I look in my task manager then it uses a lot of PC RAM , so I think yes it is running out of memory, I have updated my post if that helps. — Jigberto, Jun 13 '14 at 17:55
@Jigberto: Your edit is a separate question; revert it and ask a new question — Xan, Jun 14 '14 at 07:09

score 3 · Accepted Answer · edited May 23 '17 at 12:05

3

Your extension crashes because it runs out of memory due to your inefficient way of uploading data.

You should not load the file in memory, but pass the file object to XMLHttpRequest so that Chrome can stream the file contents in the upload form. This can be done with the FormData object.

The easiest way of doing this is to upload the form in a content script instead of a background page, because it is not easy to pass a File to the background.

// In a content script:
var myfile = document.getElementById('myfile').files[0];
var form = new FormData();
form.append('myfile', myfile);
// Add more key-value pairs using form.append(...key..., ...value...);

// Upload
var x = new XMLHttpRequest();
x.open('POST', 'http://example.com/');
x.send(form);

This is all it takes to stream files through XMLHttpRequest. The FormData object will be serialized and uploaded using the multipart/form-data algorithm. For more details, see http://www.w3.org/TR/XMLHttpRequest2/#the-send-method.

If you don't want to upload in a content script (because you want the upload to continue even after the page closes, then you have to pass the File object to the background page via a Shared Worker (that acts as a trampoline). If you want to learn more about this option, read Does chrome.runtime support posting messages with transferable objects?.

edited May 23 '17 at 12:05

Community

1
1

answered Jun 13 '14 at 18:17

Rob W

341,306
83
791
678

I was thinking about FormData, I need to try it. but I'm not sure if I must use trampoline, becuase I pass the createObjectURL with JSON.stringify and it work very well , if I understand correctly the url object is not loaded the whole file in memory, it only create a indexed path to that file to access and load. – Jigberto Jun 13 '14 at 18:31
Also, FormData it is loading the whole content before forwarding it to XMLHttpRequest or it will stream it rather than load it whole file in memory? if FormData is sending like streaming then this should avoid huge memory usege ? – Jigberto Jun 13 '14 at 18:35
@Jigberto When you use the `blob:`-method, then the memory will eventually be claimed when you convert the `blob:`-URL to a Blob for use with `FormData`. This is not ideal. When you pass a `File` directly to `FormData` and `XMLHttpRequest`, then the data will be streamed. You could in theory upload many Gigabytes at once with the method I proposed. – Rob W Jun 13 '14 at 19:10
Rob, I have accepted your answer and upvoted it because FormData is good direction for me if it is streaming rather than loading whole data in memory. But then my next question is how to send multipart form/data with FormData(), please check my next question. thanks. http://stackoverflow.com/questions/24217640/send-multipart-form-data-with-formdata – Jigberto Jun 14 '14 at 07:30
@Jigberto I had already added an example (commented) to show how additional key-value pairs can be added. And I have also provided a link to the documentation of FormData, go read it if anything is unclear... – Rob W Jun 14 '14 at 07:31
In cade it was not clear to you: Passing a FormData object already results in a multiparty/form data MIME. For the exact algorithm, read specification at the https://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#the-send()-method – Rob W Jun 14 '14 at 07:51
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/55621/discussion-between-jigberto-and-rob-w). – Jigberto Jun 14 '14 at 08:28

chrome extension crashes when i try to upload a big file

1 Answers1

Linked