Accessing already downloaded data

Question

I need to download a large (>100MB) file of data via XmlHttpRequest. The data is from a third party and I would like to display the content gradually as it gets downloaded.

So I thought the following would work:

var req = new XMLHttpRequest();
req.open( "GET", mirror.url, true );
req.responseType = "arraybuffer";

req.onload = function( oEvent ) {
    console.log( "DONE" );
};
var current_offset = 0;
req.addEventListener("progress", function(event) {
    if( event.lengthComputable ) {
        var percentComplete = Math.round(event.loaded * 100 / event.total);
    }
    var data = req.response;
    // Fails here: req.response is null till load is called

    var dataView = new DataView( data );
    while( current_offset < dataView.byteLength ) {
        // do work
        ++current_offset;
    }

    console.log( "OFFSET " + current_offset + " [" + percentComplete + "%]" );

}, false);
try {
    req.send( null );
} catch( er ) {
    console.log( er );
}

Sadly, according to the spec, .response is not available.

Is there any way to access the already downloaded data without going to such horrible workarounds like using Flash?

EDIT:

Found at least a working non-standard solution for Firefox:

responseType = "moz-chunked-arraybuffer";

See also: WebKit equivalent to Firefox's "moz-chunked-arraybuffer" xhr responseType

Wait, you want to send 100MB to my browser?! Which might be on my phone? — DOK, Mar 08 '13 at 15:17
@DOK I trust you are intelligent enough to not call a bandwidth and computation intensitive website on our phone. — abergmeier, Mar 08 '13 at 15:28

TAS · Accepted Answer · 2013-03-08T17:37:39.907

2

One solution would be to download only parts of the file using range requests to only download parts of the file. For more information see the following blog post HTTP Status: 206 Partial Content and Range Requests.

If you have control over the server you could also split up the file on the server side and download chunks of it. This way you would be able to access the response as the different chunks are recieved.

A third approach would be to use WebSockets to download the data.

If you have no control of the server you are downloading from and none of the other options will work, you will probably need to implement a proxy service that will act as an intermediate and allow you to download only part of it.

edited Mar 08 '13 at 17:37

answered Mar 08 '13 at 17:11

TAS

2,039
12
17

Sadly the data is from a third party, so I have to use the file the way they provide it. – abergmeier Mar 08 '13 at 17:18
Added two more possible solutions that might work when you don't have control of the server. – TAS Mar 08 '13 at 17:38
As long as _Chrome_ doesn't implement _Streams API_, this really seems to be the only solution. Will have to look at a partial content workaround, since I don't want to have costs for hosting. – abergmeier Mar 08 '13 at 18:14

score 0 · Answer 2 · answered Mar 08 '13 at 15:46

0

Have you looked at using a library for this?

I've never used it to load a 100MB file, but PreloadJS is pretty nice for loading all kinds of assets.

answered Mar 08 '13 at 15:46

Chris Jaynes

2,868
30
29

Is there any mention, that you can access the partially downloaded state? Cannot find it. – abergmeier Mar 08 '13 at 15:49
Hmm. I thought I remembered that you could, but perhaps you're right. It might not be possible until the data finishes loading. You can get to the XHR.response, and it is an ArrayBuffer object, but it might only be populated after load. – Chris Jaynes Mar 08 '13 at 16:48

Accessing already downloaded data

2 Answers2