10

I'm still trying to figure out how to use jQuery deferred object in recursive AJAX call. I have a code like this

function request(page, items){    

    //building the AJAX return value for JSFiddle dummy AJAX endpoint
    var ret = {
        totalPage: 10,
        currentPage: page,
        items: []
    };
    for (var i = page; i < (page + 5); i++){
        ret.items.push(i);
    }

    //calling the AJAX
    $.ajax({
        url: '/echo/json/',
        method: 'POST',
        dataType: 'json',
        data: {
            delay: 1,
            json: JSON.stringify(ret)
        },
        success: function(data){
            if (data.currentPage <= data.totalPage){
                var filtered = data.items.filter(function(el){
                    return el % 2 == 1;
                });
                var newitems = items.concat(filtered);
                console.dir(newitems);
                request(data.currentPage + 1, newitems);
            } else {
                console.dir(items);
                //resolve all item
            }
        }
    });
}

function requestAll(){
    request(1, []);
    //should return a promise tha contains all items
}

Here's the JSFiddle http://jsfiddle.net/petrabarus/BHswy/

I know how to use promise in single AJAX call, but I have no idea how to use it in a recursive AJAX call. I want to call the requestAll function in a way similar like below

var promise = requestAll();
promise.done(function(items){
    console.dir(items);
});

How can I do this?

royhowie
  • 11,075
  • 14
  • 50
  • 67
Petra Barus
  • 3,815
  • 8
  • 48
  • 87
  • 2
    I think you could still use a single promise if you did something like this: http://jsfiddle.net/gGAAy/ – Jason P Apr 29 '14 at 04:04
  • ah, i didn't know if i can do that. thanks. – Petra Barus Apr 29 '14 at 04:14
  • 1
    @JasonP: That's not really using the power of promises… – Bergi Apr 29 '14 at 04:27
  • FYI, your code isn't actually recursive because `request()` has already finished executing before you call the next one from the `success` handler. – jfriend00 Apr 29 '14 at 06:26
  • I think I still can call it as recursive because the AJAX will actually call the same function that called it before. – Petra Barus Apr 29 '14 at 08:08
  • @PetraBarus - yes, it seems conceptually like recursion, but there is no build-up on the stack frame like real recursion because the 2nd and so on call to the function occur from the asynchronous callback which occurs after the previous call has already finished and unwound it's call stack. Mostly just a terminology nuance I guess, but important in some circumstances to understand the difference. Same reason that `function go() {doSomething(); setTimeout(go, 1000);}` isn't actually recursion either. – jfriend00 Apr 30 '14 at 00:16
  • I finally understand your explanation. Nevertheless, other newbies programmer like me will probably search for `recursive` as the keyword by intuition. Thanks for the explanation. – Petra Barus Apr 30 '14 at 02:49

2 Answers2

18

You should not use the success parameter if you want to work with promises. Instead, you want to return a promise, and you want to use then to transform the results of a promise into something different, possibly even another promise.

function request(page) {    
    …
    // return the AJAX promise
    return $.ajax({
        url: '/echo/json/',
        method: 'POST',
        dataType: 'json',
        data: {
            delay: 1,
            json: JSON.stringify(ret)
        }
    });
}

function requestOddsFrom(page, items) {
    return request(page).then(function(data){
        if (data.currentPage > data.totalPage) {
            return items;
        } else {
            var filtered = data.items.filter(function(el){ return el%2 == 1; });
            return requestOddsFrom(data.currentPage + 1, items.concat(filtered));
        }
    });
}

function requestAll(){
    return requestOddsFrom(1, []);
}

requestAll().then(function(items) {
    console.dir(items);
});
Bergi
  • 630,263
  • 148
  • 957
  • 1,375
  • 1
    Actually it's quite straightforward, with the tail recursion that the OP already has it's even simpler than I initially thought. The [key to promises](http://stackoverflow.com/a/22562045/1048572) here is that the "recursively" called `requestOddsFrom(…)` will get adopted by `return`ing it to `then`, therefore making the overall result the result of the inner promise. – Bergi Apr 29 '14 at 08:17
  • It's straightforward only to someone who understands how promises link when returning a promise from a `.then()` handler. That is not a simple concept by any means (many people use promises without understanding that level of usage) and is not straightforward to follow how the code logic works or how the promises flow. Perhaps it's trivial once you fully grok it, but please have some empathy for folks who are not as advanced as you are. Your explanation of how it's quite straightforward doesn't help me understand how it works and that's what I was hoping you would explain further. – jfriend00 Apr 29 '14 at 22:24
  • Here's what I don't follow. `requestOddsFrom()` returns the value from calling `request().then()`. That's going to be a 2nd generation promise returned by the `.then()`. I get it that far. What I don't get is how N more calls to `requestOddsFrom()` gets the data back to that promise's `.done()` handler in `requestAll()`. I tried looking at the jQuery `.then()` code to see if I could follow it, but that's some of the most obtuse code I've ever seen (I think in the interest of making it small) that I could not follow its logic. The jQuery doc was of no help on this topic either. – jfriend00 Apr 29 '14 at 22:29
  • @jfriend00: Maybe you don't understand what that "2nd generation promise" means. It is a promise that will be resolved with the result of the `requestOddsFrom(…)` recursive call promise that is returned from the `then` callback. Or it will just resolve with the `items` that are returned from the `then` callback in case that there are no more pages. – Bergi Apr 29 '14 at 23:10
  • Yeah, I don't understand how N recursive calls to `requestOddsFrom()` gets the data back to the first promise that was returned from the original call to `requestOddsFrom()` because that's the promise that `requestAll().done()` is using. I don't understand how that works. – jfriend00 Apr 29 '14 at 23:32
  • Well, that's the "magic" of `then`, and the [point of promises](http://domenic.me/2012/10/14/youre-missing-the-point-of-promises/) being monadic. Consider `Promise.for("a").then(function(x) { return Promise.for(x+"b"); }).done(console.log)` - it will log `ab`. – Bergi Apr 29 '14 at 23:41
  • That's a [good article reference](http://domenic.me/2012/10/14/youre-missing-the-point-of-promises/) - thanks. One thing I don't understand from that article is that it talks about exceptions propagating through promises (causing them to fail and that failure to propagate like it would in synchronous code). When I look at the jQuery 2.1.0 source for `.then()` there are no exception handlers there so I don't see how it can be catching an exception in a then callback and turning it into a failed propagated promise. Am I missing something or is jQuery missing something? – jfriend00 Apr 30 '14 at 00:01
1

Since you're already sequencing the Ajax operations one after the other, without totally restructing your code, you can just use one deferred that you resolve on the last Ajax call:

function request(page, items, defer){    

    //building the AJAX return value for JSFiddle dummy AJAX endpoint
    var ret = {
        totalPage: 10,
        currentPage: page,
        items: []
    };
    for (var i = page; i < (page + 5); i++){
        ret.items.push(i);
    }

    //calling the AJAX
    $.ajax({
        url: '/echo/json/',
        method: 'POST',
        dataType: 'json',
        data: {
            delay: 1,
            json: JSON.stringify(ret)
        },
        success: function(data){
            if (data.currentPage <= data.totalPage){
                var filtered = data.items.filter(function(el){
                    return el % 2 == 1;
                });
                var newitems = items.concat(filtered);
                console.dir(newitems);
                request(data.currentPage + 1, newitems, defer);
            } else {
                console.dir(items);
                //resolve the deferred
                defer.resolve(items);
            }
        }
    });
}

function requestAll(){
    var deferred = jQuery.Deferred();
    request(1, [], deferred);
    return deferred.promise();
}

requestAll().done(function(items) {
    // all ajax calls are done
});

OK, after much new promise learning, here's a fully promise version that makes use of promise chaining (returning a promise from a .then() handler). Concepts borrowed and learned from Benji's implementation, but this is organized a bit differently and commented for learning (it would actually be quite short without comments and without the dummy Ajax call stuff):

function requestPages(startPage, endPage) {

    function request(page, items){    
        // building the AJAX return value for 
        // JSFiddle dummy AJAX endpoint
        var ret = {
            currentPage: page,
            items: []
        };
        for (var i = page; i < (page + 5); i++){
            ret.items.push(i);
        }

        // Do Ajax call, return its promise
        return $.ajax({
            url: '/echo/json/',
            method: 'POST',
            dataType: 'json',
            data: {
                delay: 1,
                json: JSON.stringify(ret)
            }
        }).then(function(data) {
            // mock filter here to give us just odd values
            var filtered = data.items.filter(function(el){
                return el % 2 == 1;
            });
            // add these items to the ones we have so far
            items = items.concat(filtered);

            // if we have more pages to go, then do the next one
            if (page < endPage){
                // Advance the currentPage, call function to process it and
                // return a new promise that will be chained back to the 
                // promise that was originally returned by requestPages()
                return request(page + 1, items);
            } else {
                // Finish our iteration and 
                // return the accumulated items.
                // This will propagate back through 
                // all the other promises to the original promise
                // that requestPages() returned
                return(items);
            }
        });
    }    

    // call the first request and return it's promise    
    return request(startPage, []);
}

// request pages 1 through 10 inclusive
requestPages(1, 10).done(function(items) {
    // all ajax calls are done
    console.log(items);
});

Working jsFiddle: http://jsfiddle.net/jfriend00/pr5z9/ (be patient, it takes 10 seconds to execute for 10 Ajax calls that each take 1 second).

One issue I noticed about this version is that because it only uses the promises created by $.ajax(), the code cannot do a .notify() to trigger progress notifications. I found that I wanted to trigger a progress notification on the originally returned promise as each Ajax call completed, but without creating my own Deferred, I couldn't do that because you can't do a .notify() on a promise, only on a Deferred. I'm not sure how to solve that and keep with Benji's architecture of not creating/resolving your own deferred.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • Why the downvote? This does exactly what the OP asked for. It also makes the fewest changes to the OP's existing code. – jfriend00 Apr 29 '14 at 05:43
  • 1
    It's less recoverable, does not use promises correctly and is longer than Bergi's solution. Also, the recursion is on a much longer function. Promises already have continuation built out of the box, this sort of callback recursion without return values is unnecessary here. Also, it would provide worse stack traces if a real promise library was used. See https://github.com/petkaantonov/bluebird/wiki/Promise-anti-patterns#the-deferred-anti-pattern – Benjamin Gruenbaum Apr 29 '14 at 05:43
  • @BenjaminGruenbaum - How does it not use promises correctly? Bergi's solution conveniently omits a bunch of the OP's required code (replaced with a ...) so I don't know about the length. Also, why downvote a correct solution. Just upvote another one you like. You can have your own favorite architecture if you want, but this saves rewriting the entire OP's code and it does exactly what the OP asked for. Not generally deserving of a downvote. There is no actual recursion here either so I don't know what you're saying about stack traces. – jfriend00 Apr 29 '14 at 05:46
  • Downvotes are not for correctness, they are for usefulness proper - this is how I vote and the only way I vote. I find your solution not useful given Bergi's existing one, and it's encouraging an anti pattern the promise user community is trying to eliminate (for example, your code is not throw safe (big deal), it gives less debug information, it doesn't use promises other than for the return value, and it's less clear). I'm sorry that you don't appreciate my voting, but I think you'll manage that -2 given all the times I've up-voted your useful answers in the past. – Benjamin Gruenbaum Apr 29 '14 at 05:51
  • @BenjaminGruenbaum - OK, your opinion - not what I use downvotes for though. I'd be curious which solution the OP finds easier to understand and follow the logic for. I found Bergi's complicated to follow (it uses some pretty advanced concepts which are not necessarily straightfoward to follow for the non-promise-expert) which is why I offered an answer that is a simple modification to the OP's code and very easy to follow. – jfriend00 Apr 29 '14 at 05:59
  • I think given the (tens of) upvotes I've given your answers in the past we can manage this disagreement. As for "easy", http://www.infoq.com/presentations/Simple-Made-Easy :) – Benjamin Gruenbaum Apr 29 '14 at 06:41
  • @BenjaminGruenbaum - there is no way you can say that Bergi's solution is simple to understand how it works unless you are an advanced promise expert (I don't even follow it yet and I consider myself at least an intermediate promise user). It would help if Bergi explained how it works. – jfriend00 Apr 29 '14 at 07:01
  • 1
    I like Bergi's code better. It looks cleaner and more modular, because I can modify the filtering function easily. And the idea of transforming the result of a promise into another promise is exactly what was in my mind. My question was a bit of pretense because at that moment I had no clear idea what I wanted to ask. The removed code is actually just a dummy AJAX endpoint for JSFiddle (so it's okay to remove it). – Petra Barus Apr 29 '14 at 08:07
  • @PetraBarus - as best I can tell, we both have the same filter function copied from you. He just collapsed it to one line - I left it as you had it. I'm fine if you like Bergi's code better - I took your architecture and added what you asked for changing as little as needed - Bergi rearchitected the flow. That's all fine. Just make sure you understand exactly how whatever code you go with works. – jfriend00 Apr 29 '14 at 23:30
  • Yeah, Bergi's answer seems to be correct in the JSFiddle, but when I tried reproducing it using a bit different pattern, it seems to fail. And I ended up using yours. Pretty straight forward but correct in that different pattern. Maybe it's because I still don't understand how Bergi's code works. I'm going to write down the JSFiddle for the new pattern I used and ask Bergi for that. Thanks a lot for your answer. – Petra Barus Apr 29 '14 at 23:36
  • @BenjaminGruenbaum - FYI, it turns out that Bergi's code isn't "throw safe" either because jQuery promises don't catch exceptions - an artifact of jQuery's partial promise implementation which this code is forced to use because of `$.ajax()`. I'm not arguing against Bergi's design, just pointing out that it doesn't accrue that advantage in jQuery. – jfriend00 Apr 30 '14 at 00:21
  • @PetraBarus - As my brain slowly groks the concept of returning a new promise from a `.then()` handler, I've now added an implementation that works similar to Benji's (though organized and commented in my own way). – jfriend00 Apr 30 '14 at 02:42