XmlHttpRequest.responseText while loading (readyState==3) in Chrome

Question

I am trying to "streaming" (from server to client) in Javascript by ajax (by XmlHttpRequest (=xhr). I am using modified handleResponse function described in Cross-browser implementation of "HTTP Streaming" (push) AJAX pattern

function handleResponse() {
if (http.readyState != 4 && http.readyState != 3)
    return;
if (http.readyState == 3 && http.status != 200)
    return;
if (http.readyState == 4 && http.status != 200) {
    clearInterval(pollTimer);
    inProgress = false;
}
// In konqueror http.responseText is sometimes null here...
if (http.responseText === null)
    return;

while (prevDataLength != http.responseText.length) {
    if (http.readyState == 4  && prevDataLength == http.responseText.length)
        break;
    prevDataLength = http.responseText.length;
    var response = http.responseText.substring(nextLine);
    var lines = response.split('\n');
    nextLine = nextLine + response.lastIndexOf('\n') + 1;
    if (response[response.length-1] != '\n')
        lines.pop();

    for (var i = 0; i < lines.length; i++) {
        // ...
    }
}

if (http.readyState == 4 && prevDataLength == http.responseText.length)
    clearInterval(pollTimer);

inProgress = false;
}

With php script, which flushes me data (without ajax it really flushes data to browser while progressing)

I have no problem in Firefox, but Google Chrome and IE give me an empty responseText while xhr.readyState equals to 3. I found that problem described in the Internet, but it didn't give me any solution.

Do you know, how to pass by this implementation problem in Chrome? (w3c says, that responseText can't be NULL in readyState==3 - Chrome implemented this rule, but gives only empty string)

And if you don't know, do you know any working solution in some products? (opensource frameworks, librararies etc.)

Thanks a lot for your ideas.

Edit: The workaround is in creating iframe, call the script to iframe and flush data here and grab data by javascript from iframe. But this is not ajax solution. I really would like to see pure ajax solution.

Answer 1

Chrome has a bug where it will only populate xhr.responseText after a certain number of bytes has been received. There are 2 ways to get around this,

Set the content type of the return to "application/octet-stream"

or

Send a prelude of about 2kb to prep the handler.

Either of these methods should make chrome populate the responseText field when readyState == 3.

IE7/8 on the other hand can't do it, you need to resort to long polling or use the cross domain trick with XDomainRequest in IE8, a la MS

Answer 2

To expand on Andrew's answer, this is the cross-browser solution I came up with.

Works correctly in 99% of browsers, namely IE ≥ 8, Chrome, Firefox, and Safari, sending incremental events as soon as data is received by the browser (but see the notes below.)

if (/MSIE [8-9]/.test(navigator.appVersion)) {
    var get = new XDomainRequest()
    get.onprogress = handleData
    get.onload = handleData
} else {
    var get = new XMLHttpRequest()
    get.onreadystatechange = handleData
}
get.open('get', '/example/url')
get.send()

function handleData() {
    if (get.readyState != null && (get.readyState < 3 || get.status != 200)) {
        return
    }
    // process incremental data found in get.responseText
}

IE 8–9 will start populating responseText after 2kB of data, so if that's not ok, you should send an initial padding of 2kB.

Chrome needs either that, or Content-Type: application/octet-stream.

Answer 3

Have you considered using WebSockets or server-sent events?

Most major browsers now support the WebSocket protocol, though if your site needs to work in IE 9 or older, or in Android Browser 4.3 or older, you would have to keep the code that uses XMLHttpRequest as a fallback.

Most of these browsers also support a feature called server-sent events, which unlike WebSockets, can be implemented on the server using a traditional HTTP daemon and CGI/PHP script, though only provides one-way communication.

See also: WebSockets vs. Server-Sent events/EventSource

Answer 4

Well, unfortunately every part of XmlHttpRequest (or any web standards) isn't fully implemented in all browsers. But you have several other options for HTTP Streaming:
Wikipedia: Push technology
Wikipedia: Comet (programming) Wikipedia: Web Sockets (experimental, low browser support)

I saw in your comment that you would like it to be pure AJAX, but I like to suggest possible alternate ways to solutions. You could use a JavaApplet where possible or a Flash Object. For the latter you won't need a flashy and expensive IDE, you can use Haxe to create Flash/SWF files and you will feel pretty comfortable with it as you know JavaScript.

Here is a Flash/Neko chat example that probably can be adopted to other platforms and usages as well.

I wish you best of good luck.

Answer 5

try using the responseStream/responseBody property when in IE. I was thinking of doing a similar thing once and ran into the same problem. Unfortunately neither are w3c specs

http://www.w3.org/TR/XMLHttpRequest/#the-responsetext-attribute

Answer 6

As i understand it, making partial text available on readyState 3 is a non-standard firefox only behaviour that is simply not possible to directly emulate in other browsers, what you might want to do instead is make multiple sequential requests for small chunks of the data rather than one 'streaming' request

Answer 7

This worked for me for Chrome, but not IE:

[test.php]:

<?php
Header('Content-type: text/plain');
while (1) {
    echo str_pad('test: '.mt_rand(1000,9999), 2048, ' ');
    flush();
    sleep(1);
}

[test.html]:

<!DOCTYPE html>
<html lang="en">
    <head>
        <title>Stream test</title>
        <meta charset="UTF-8" />
        <script type="text/javascript">
            function xmlHttpRequest() {
                return (function (x,y,i) {
                    if (x) return new x();
                    for (i=0; i<y.length; y++) try { 
                    return new ActiveXObject(y[i]);
                    } catch (e) {}
                })(
                    window.XMLHttpRequest, 
                    ['Msxml2.XMLHTTP','Microsoft.XMLHTTP']
                );
            };
            function stream(url) {
                // Declare the variables we'll be using
                var xmlHttp = xmlHttpRequest();
                xmlHttp.open("GET", url, true);
                var len = 0;
                xmlHttp.onreadystatechange = function() {
                if (xmlHttp.status == 200 && xmlHttp.readyState >=3) {
                    var text = xmlHttp.responseText;
                    text = text.substr(len, text.length-len);
                    len = xmlHttp.responseText.length;
                    console.log(text);
                }
                }
                xmlHttp.send(null);
            }           
            stream('/test.php');
        </script>
    </head>
    <body>
    </body>
</html>

YMMV.

Answer 8

As Jaroslav Moravec said, if you set the content-type in the header of the stream to application/x-javascript it works in Safari, Chrome and Firefox.

I have not tested IE.

Answer 9

Setting the content type of the return to "application/octet-stream" as suggested by Andrew was a great solution. Plus you should use XDomainRequest on IE.

To read the data as they come, you should just use an infinite loop (that stops when readystate = 4 or XDomainRequest.onLoad has been called) with a timeout.

Here's how I would do it:

var i = 0;
var returnValue = function() {
    if (!finished) {
        setTimeout(returnValue, 100);
    }
    var resp = ajax.responseText;
    var newI = resp.lastIndexOf(";") + 1;
    if (newI > i) {
        var lines = resp.substring(i, newI).split(";");
        for (var x = 0; x < lines.length; x++) {
            eval(lines[x]);
        }
        i = newI;
    }
}

Note: some says using eval is risky, I claim that's not where any risk really come from.

Answer 10

The URLyou are submitting the request to - is it part of your domain?

It coulld be because of the same origin policy.

See this question and possible ways to circumvent it (and this article).

Answer 11

once i had this problem using safari (never tested with chrome, maybe there was the same problem (chrome/safari both use the same rendering-engine (as far as i know) - don't know about the js-parts)). i never found a solution to work around that, but because of it was a small app in a company-wide intranet it wasn't a big problem to not support safari (ff was the default-browser anyway, and ff works).