39

I'm using play framework, to generate chunked response. The code is:

class Test extends Controller {
    public static void chunk() throws InterruptedException {
        for (int i = 0; i < 10; i++) {
            String data = repeat("" + i, 1000);
            response.writeChunk(data);
            Thread.sleep(1000);
        }
    }
}

When I use browser to visit http://localhost:9000/test/chunk, I can see the data displayed increased every second. But, when I write a javascript function to receive and handle the data, found it will block until all data received.

The code is:

$(function(){
    $.ajax(
        "/test/chunked", 
        {
            "success": function(data, textStatus, xhr) {
                alert(textStatus);
            }
        }
    );
});

I can see a message box popped up after 10s, when all the data received.

How to get the stream and handle the data in time?

Pacerier
  • 86,231
  • 106
  • 366
  • 634
Freewind
  • 193,756
  • 157
  • 432
  • 708

3 Answers3

67

jQuery doesn't support that, but you can do that with plain XHR:

var xhr = new XMLHttpRequest()
xhr.open("GET", "/test/chunked", true)
xhr.onprogress = function () {
  console.log("PROGRESS:", xhr.responseText)
}
xhr.send()

This works in all modern browsers, including IE 10. W3C specification here.

The downside here is that xhr.responseText contains an accumulated response. You can use substring on it, but a better idea is to use the responseType attribute and use slice on an ArrayBuffer.

adius
  • 13,685
  • 7
  • 45
  • 46
phil pirozhkov
  • 4,740
  • 2
  • 33
  • 40
  • 2
    why is using of ArrayBuffer better? – 4esn0k Feb 04 '13 at 02:21
  • @4esn0k: for example if you decide to use a binary protocol, which is a good idea. Another point here is that browser doesn't have to keep the entire data in memory, and as far as i understand, it's easier to achieve with `ArrayBuffer` than with a `String`. `slice` should work better, since it doesn't have to work with UTF-8 String as `substring` does, but can just use array indexes. – phil pirozhkov Feb 04 '13 at 14:03
  • @4esn0k `moz-chunked-text` looks very promising, but is only available in FF and unfortunately isn't a part of XMLHttpRequest2 spec, so it's unlikely to be implemented in other browsers sometimes soon. – phil pirozhkov Feb 06 '13 at 02:12
  • 1
    is there a more structured approach / better library to do that? – Edmondo Apr 08 '13 at 09:29
  • @Edmondo1984 Take a look at [this](http://stackoverflow.com/questions/12275128/jquery-ajax-extension-xdomainrequest-onprogress) – phil pirozhkov Apr 09 '13 at 08:33
  • 2
    @Edmondo1984, I ended up creating this package which allows more efficient tranfer than jsonpipe: https://www.npmjs.com/package/chunked-request – JonnyReeves Mar 22 '16 at 18:44
  • @JonnyReeves do you have benchmarks for chunked-request vs. jsonpipe? What makes it more efficient - the ability to pass in a custom chunk parser? – Aaron Jul 01 '16 at 02:06
  • 1
    I am trying to do the same thing as OP asked. I followed Phil's directions and getting the response in chunks. But What should I do to NOT accumulate response? I tried to set responseType to arraybuffer but then I do not get any response. – NetDeveloper Feb 17 '17 at 16:29
  • I found it helpful to wrap the xhr request in an ajax request because rails automatically includes the CSRF token in AJAX requests. – Wylliam Judd Oct 19 '17 at 00:17
  • @philpirozhkov Can you please provide an example with `responseType: 'arraybuffer'` ? – rinat.io Dec 15 '17 at 10:22
  • @philpirozhkov Just checked [specification](https://xhr.spec.whatwg.org/#the-response-attribute) and it seems like `response` will be `null` until xhr is done in case `responseType = "arraybuffer"` – rinat.io Dec 15 '17 at 10:30
7

Soon we should be able to use ReadableStream API (MDN docs here). The code below seems to be working with Chrome Version 62.0.3202.94:

fetch(url).then(function (response) {
    let reader = response.body.getReader();
    let decoder = new TextDecoder();
    return readData();
    function readData() {
        return reader.read().then(function ({value, done}) {
            let newData = decoder.decode(value, {stream: !done});
            console.log(newData);
            if (done) {
                console.log('Stream complete');
                return;
            }
            return readData();
        });
    }
});
rinat.io
  • 3,168
  • 2
  • 21
  • 25
0

the success event would fire when the complete data transmission is done and the connection closed with a 200 response code. I believe you should be able to implement a native onreadystatechanged event and see the data packets as they come.

kishu27
  • 3,140
  • 2
  • 26
  • 41