6

I have to make a sequence of fetch() Promise: I have only 1 url at a time, this means only 1 fetch() promise. Every time I receive a json,this one contains an url for another json, so I have to make another fetch() promise.

I'm able to work with multiple promise, but in this case I can't do Promise.all(), because I don't have all the url, but only one.

This example doesn't work, it all freezes.

function fetchNextJson(json_url) 
{
    return fetch(json_url, {
            method: 'get'
        })
        .then(function(response) {
            return response.json();
        })
        .then(function(json) {
            console.log(json);
            return json;
        })
        .catch(function(err) {
            console.log('error: ' + error);
        });
}


function getItems(next_json_url) 
{
    if (!(next_json_url)) return;

    get_items = fetchNextJson(next_json_url);

    interval = $q.when(get_items).then(function(response) {
        console.log(response);
        next_json_url = response.Pagination.NextPage.Href;
    });

    getItems(next_json_url);
}


var next_json_url = 'http://localhost:3000/one';

getItems(next_json_url);
FrancescoN
  • 2,146
  • 12
  • 34
  • 45
  • What is expected behavior and overall goal? – charlietfl Jun 26 '16 at 02:12
  • How many total requests are expected to be processed? `$q.when()` does not appear to be necessary. Why do you call `getItems(next_json_url)` outside of `.then()` within `getItems()` call? – guest271314 Jun 26 '16 at 02:12
  • what is the base case? when will you stop fetching data? – Zohaib Ijaz Jun 26 '16 at 02:12
  • i have to download 1 json at a time: the first has a reference for the next one, and so on @guest27131 as many as I want.. 1-40. I'll stop when there is no more reference, in current json, for the next json – FrancescoN Jun 26 '16 at 02:21
  • But does something have to happen only after all data is received? Or can you do like $resource does and return an empty array that continues to populate while requests are being made? Not clear what you are doing with all the results of these requests – charlietfl Jun 26 '16 at 02:24
  • When I receive the json I have to save it in localstorage, yes something has to happen. – FrancescoN Jun 26 '16 at 02:27
  • Save as one localstorage key for all data or individual keys for each response? And how does it impact the rest of the app? – charlietfl Jun 26 '16 at 02:28
  • Individual key, but this isn't a problem, offline navigation, 15kb for json. Please it's important, do you know how to achieve the goal? – FrancescoN Jun 26 '16 at 02:33
  • So if this is for offline navigation using individual storage keys why can't you just store each response as it is received? Still haven't really isolated what specific roadblock is – charlietfl Jun 26 '16 at 02:35
  • User will download and store Pages to view offline: each page is a json combined with Angular. – FrancescoN Jun 26 '16 at 02:43
  • OK... so that means nothing critical has to happen only after **all** results received. Makes it much simpler – charlietfl Jun 26 '16 at 02:45
  • 1
    Might consider making sure you have a master nav map and work from that instead. Would make it easier to handle errors since one error will break your chain and map would give you a place to track updates from – charlietfl Jun 26 '16 at 03:44
  • an error happening inside promise.all is a thing I was thinking about, because if a Promise fails, all the chain fails if I'm right. So, what is this map? Do you have a doc/ tutorial link? Thanks, thanks – FrancescoN Jun 26 '16 at 13:37

1 Answers1

9

You can use recursion

function fetchNextJson(json_url) {
    return fetch(json_url, {
            method: 'get'
        })
        .then(function(response) {
            return response.json();
        })
        .then(function(json) {
            results.push(json);
            return json.Pagination.NextPage.Href 
                   ? fetchNextJson(json.Pagination.NextPage.Href)
                   : results
        })
        .catch(function(err) {
            console.log('error: ' + error);
        });
}


var next_json_url = 'http://localhost:3000/one';
var results = [];

fetchNextJson(json_url).then(function(res) {
  console.log(res)
})
guest271314
  • 1
  • 15
  • 104
  • 177
  • Took me a few minutes to wrap my head around why this works so well... conclusion is constantly adding `then()` to chain until finally a non promise is returned. Sound about right? – charlietfl Jun 26 '16 at 03:09
  • @charlietfl Yes. If `json.Pagination.NextPage.Href` is defined, call `fetchNextJson`, else return `results` array – guest271314 Jun 26 '16 at 03:11
  • 2
    Made simplified demo using a counter instead of having a new url to return http://plnkr.co/edit/5boVimpYnoE1o41FXZGG?p=preview – charlietfl Jun 26 '16 at 03:12
  • 1
    @charlietfl _"conclusion is constantly adding then() to chain until finally a non promise is returned."_ Any result returned from `.then()` chained to `fetchNextJson` should be promise after first call; `.then()` either recursively calls `fetchNextJson` or returns accumulated results, or other value, as a promise value – guest271314 Jun 26 '16 at 03:19
  • Tricky part i guess is how to manage going back to fill holes that hit catch in `fetchNextJson()` – charlietfl Jun 26 '16 at 03:23
  • 1
    @charlietfl `.catch()` should stop recursion, unless `fetchNextJson` is called within `.catch()`. One approach could be to handle error, then return `fetchNextJson` at `.then()` chained to `.catch()` or in `.catch()`; though given present requirement is to call next `fetchNextJson` with results from current `fetchNextJson` there would not appear to be any further tasks to perform - unless url is provided from an optional source – guest271314 Jun 26 '16 at 03:24
  • Right ... but case of say an intermittent connection during one request that fails could try returning `fetchNextJson()` at least once for that particular url. Trying to wrap head around that strategy more still. Or set timer to try again say in a minute – charlietfl Jun 26 '16 at 03:28
  • 1
    Had not considered such a case. Yes, could call `fetchNextJson()` with current url within `.catch()` if error occurred, to try to continue recursive chain; store, limit number of reties. Though, again, given present requirement, there would not be a source for next url. If requirement included seaparate array of urls, could iterate entire array, whether errors were caught, handled, or successful response returned. Once error handled by `.catch()`, can return promise to chained `.then()` that should not be an error within `.then()`; could continue with recursive call to function – guest271314 Jun 26 '16 at 03:32
  • Yeah ... i think creating all that fallback strategy would take some tinkering – charlietfl Jun 26 '16 at 03:33
  • One other scenario can't figure out is presumably there are more than one page that doesn't have next url... so first one would break recursion also. Sure seems like be better for OP to keep a master nav map – charlietfl Jun 26 '16 at 03:36
  • @charlietfl _"presumably there are more than one page that doesn't have next url... so first one would break recursion also"_ Yes, this appears to be expected result; perhaps only results given guidance as to present requirement from OP _"I'll stop when there is no more reference"_ in response to _"How many total requests are expected to be processed?"_ – guest271314 Jun 26 '16 at 03:38
  • 1
    Oh I agree. Just thinking out of the box a bit. – charlietfl Jun 26 '16 at 03:39
  • @charlietfl See also http://stackoverflow.com/questions/28131082/jquery-ajax-prevent-fail-in-a-deferred-sequential-loop , http://stackoverflow.com/questions/35042068/why-is-onrejected-not-called-following-promise-all-where-promise-reject-incl – guest271314 Jun 26 '16 at 03:45
  • guys thanks to you all: this was impossible to perform by me. It's a luck StackOverflow exists and all you replied me. Just to resume about error handling: let's assume I want to fetch for a certain url up to 3 times, in case of error. What if I get an error inside the `.catch()`? @guest271314 you have told about using `.then()`,where it starts another fetch retry, and .`catch()`, where it shoutout about the error, inside the first `.catch()` if I've understood. Is it right? can you give me an example? – FrancescoN Jun 26 '16 at 14:20
  • 1
    @FrancescoN See http://plnkr.co/edit/wkzEbNZ98m3zbSXMfFX1?p=info . Note [Why does fetch() work this way?](https://www.tjvantoll.com/2015/09/13/fetch-and-errors/) _"Per [MDN](https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API/Using_Fetch#Checking_that_the_fetch_was_successful), the fetch() API only rejects a promise when a “network error is encountered, although this usually means permissions issues or similar.” Basically fetch() will only reject a promise if the user is offline, or some unlikely networking error occurs, such a DNS lookup failure."_ – guest271314 Jun 26 '16 at 14:44