17

I'm trying to stream a large amount of binary data in JavaScript, accessing the data before the download completes. In most mainstream browsers, I can use the charset=x-user-defined trick to manually get the raw byte data during the progress event.

In Internet Explorer, however, this trick doesn't work and I'm left with using the VBArray(responseBody).toArray() method instead, which is painfully slow. However, since I only need to support IE 11 and later, I should be able to make use of IE's MSStream to get the data progressively. The following code works fine on IE 11 desktop, but not on a Lumia Windows Phone 8.1 device running IE 11 mobile:

var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.responseType = 'ms-stream';
xhr.onreadystatechange = function () {
    if (xhr.readyState === 3 && xhr.status === 200) {
        // reader is an MSStreamReader object
        reader.readAsArrayBuffer(xhr.response);
    }
};
xhr.send();

On the Windows Phone device, the readyState never goes past 1 and the status is 0, indicating an unknown error occurred even though no actual error is thrown.

Does anyone have any idea why this isn't working for me, or maybe a solution to the problem?

Andy E
  • 338,112
  • 86
  • 474
  • 445
  • Hmm, from the material I've found [here](https://msdn.microsoft.com/en-us/library/hh772328(v=vs.85).aspx), that should work. Is the code which generates the URL building the correct one? Also, is the `readAsArrayBuffer` method pulling the `IInputStreamObject` out of the `MSStream` via `msDetachStream()`? That's what it shows to do, but I'm not sure if that will completely terminate the rest of the stream and just give you whatever's downloaded currently or not. Are there other methods on `MSStream` to check out? – Shotgun Ninja May 11 '15 at 19:05
  • Oh, you're using `MSStreamReader` [as described here](https://msdn.microsoft.com/en-us/library/hh772312(v=vs.85).aspx). Uh... that seems to eliminate what I said earlier. Again, is your URL formed differently between Windows Phone and desktop? – Shotgun Ninja May 11 '15 at 19:50
  • @shotgun the page and the data is exactly the same on both desktop and phone, and I've tried a small separate test on both and got the same results. – Andy E May 11 '15 at 21:08
  • [This page](https://msdn.microsoft.com/en-us/library/hh673569(v=vs.85).aspx) talks about a different way to do this, under the heading "Comet Streaming". It links [here](http://go.microsoft.com/fwlink/p/?LinkId=234112), though I can't access that link currently. It looks like that's for multipart requests, though, instead of streaming... – Shotgun Ninja May 11 '15 at 21:31
  • What sort of resource are you requesting via `url`? Does that resource have browser-detection built in? If so, Windows 8+ and IE10/11 Mobile don't seem to play nice with browser-detection scripts, as far as I've seen from a few cursory Google searches. – Shotgun Ninja May 11 '15 at 21:41
  • @shotgun unfortunately, I'm not aware of any way of accessing the raw bytes in IE from the responseText property. Other browsers allow you to specify `x-user-defined` as the charset, but IE doesn't support it. It supports the responseBody property, but the roundtrip conversion of *every* byte on *every* progress event means that the conversion process blocks the thread from running the code that does something with the data. – Andy E May 11 '15 at 21:44
  • Would there be any difference between using `onprogress` and `onreadystatechange`? Your code above uses `onreadystatechange`, but `onprogress` is shown [here](https://msdn.microsoft.com/en-us/library/hh673569(v=vs.85).aspx#code-snippet-5) and it says it fires at server-defined intervals. Might those be infrequent enough to avoid hanging the process? – Shotgun Ninja May 11 '15 at 21:54
  • @AndyE I had this same problem and it had to do with cross-site domain request. Is the URL you are using accessing another domain in any way ? Also, what is desktop OS that worked when you tested it on desktop? – Tech Savant May 12 '15 at 00:01
  • @Notorious: I tested on Windows 7 with IE 11. It *is* cross domain (the target URL is on our CDN) but won't be when the code is in a production environment. If you're right and it works for same origin requests then I could live with the slowness seen in testing. I'm going to run some tests in the production environment and I'll let you know. – Andy E May 12 '15 at 07:59
  • If you test on desktop Win8/8.1 and have probs with IE or chrome, that's what I had too. And windows Phone 8.1 as you described in your question. I'm racking my brain to remember the solution, this one jammed me up for days, maybe weeks before I finally got it working. Something to do wtih MS new security policy in 8.1 – Tech Savant May 12 '15 at 08:10
  • If you are running apache and can setup mod_proxy to create a reverse proxy on the server that might work. Other things I remember trying were, of course, cors, jsonp, document.domain, window.postmessage, I tried everything under the sun, something fixed it. Curse my memory. I'm going through my code archive now trying to find that code. – Tech Savant May 12 '15 at 08:24
  • @AndyE Posted a couple small code tweaks in my answer. They might help. Couldn't hurt to try. – Tech Savant May 12 '15 at 08:48
  • @AndyE what URL are you sending request to ? – ProllyGeek May 13 '15 at 04:26
  • I would test by removing your own ajax implementation and instead use jQuery's ajax functions. That way you'll know if the problem comes from you code or some IE-specific weirdness. – frenchie May 13 '15 at 04:26
  • @AndyE What language is the ajax script in ? If it's PHP, can you check the script and make sure there is no closing tag (?>) I know it seems stupid but this has fixed status: 0 problems for me in the past. – Tech Savant May 13 '15 at 04:28
  • @NotoriousPet0: it's a static binary file, so no PHP weirdness. – Andy E May 13 '15 at 09:17
  • @ProllyGeek: I've tried several different URLs across different domains and file types. If I thought the URL was relevant to the question, I would have included it. – Andy E May 13 '15 at 10:03
  • Did you try this [VBScript hack](http://stackoverflow.com/a/1926163/112731)? If done right, it's said to work reasonably fast. – Onur Yıldırım May 13 '15 at 19:55
  • @Onur: unfortunately I can't use that because IE 11 doesn't support VBScript in edge mode. – Andy E May 15 '15 at 14:26

1 Answers1

2

Assuming you have tried same origin policy solutions and are fairly certain that the issue is not related to same origin policy....

I think the problem is IE is not getting to readystate 3 because it doesn't get that until the entire response is received.

A workaround for this problem is to send down a two kilobyte “prelude” at the top of the response stream—my test simply sends 2kb of spaces. After that initial block is received, the onprogress event is fired as each subsequent block is received from the network.

Also, have you tried this ... 

var xhr = new XMLHttpRequest();
xhr.open('GET', url, true);
xhr.responseType = 'blob';
xhr.onreadystatechange = function () {
    if (xhr.status === 200) {
        var blob = this.response;
        reader.readAsArrayBuffer(blob);
    }
};
xhr.send();
Tech Savant
  • 3,686
  • 1
  • 19
  • 39
  • Unfortunately, it still doesn't work for same domain requests. The semi-colon is fine at the end of the function expression (because it's an expression) and most linters pull you up for it not being there. Comet streaming is the method I'm currently using, but the only way to get the raw bytes is with the following trick: `new VBArray(xhr.responseBody).toArray()`. This is required for the transformation to an ArrayBuffer during each progress event and becomes slower as more data is downloaded. – Andy E May 12 '15 at 09:47
  • @AndyE Added another thing you might try with blob type. – Tech Savant May 13 '15 at 04:23
  • the `status` property is always `0`, so that doesn't work either. The more I investigate the issue, the more I'm convinced that Windows Phone 8.1 has a bug that prevents 'ms-stream' being usable as a `responseType` XHR value. – Andy E May 13 '15 at 08:28
  • Right , so if you change it to blob, you might have more luck. Also, test this on Desktop Win 8.1 under IE, you might find it also has problems there. – Tech Savant May 13 '15 at 08:29
  • When I have winphone 81 issues, i almost always see them in desktop win81 as well, so if you are not seeing them on desktop win81, then you may be right about the winphone bug – Tech Savant May 13 '15 at 08:31
  • The 2k prelude isn't an option as I don't control the environment serving the binary file. The blob response isn't usually available until the request has fully completed, but I will check Microsoft's implementation and see if it differs in that it's available at any time during the request. I only have access to a Windows Phone during office hours and I'm quite busy with other work too so I'm not sure when exactly I'll be able to test it but it will probably be today. Thanks for all your help so far. – Andy E May 13 '15 at 09:38
  • I read that the arrayBuffer can't handle non-aligned data. You may need to implement data padding to accomodate. – Tech Savant May 13 '15 at 09:42
  • It's not the green tick, but I appreciate the time spent trying to help so the bounty is yours. Thanks again. – Andy E May 15 '15 at 15:17