2

Problem Overview: I have a GreaseMonkey/TamperMonkey script in FireFox that sends several GM_xmlhttpRequest (essentially similar to a xml http request, however this wrapper allows ease in cross-origin requests) to URL's to retrieve simple JSON data. Some of the JSON data retrieved is super simple, like so,

    {id:123459876, code:"frog"}
    [{value:"water", contents:0}]

This leads me to my outline of what code I've got down for my programming task so far, and, eventually shapes what I'll ask in the end as my question.

Problem Outline: Since the workflow I'm essentially doing is as follows; (based off this SO question/answer)

  1. Send a 'GET' request with GM_xmlhttpRequest inside a Promise object
function makeRequest(reqMethod, navURL) {
    return new Promise(function (resolve, reject) {
        GM_xmlhttpRequest({
            method: reqMethod,
            url:    navURL,
            onload: function () {
                if (this.status >= 200 && this.status < 300) {
                    resolve(this.response);
                } else {
                    reject({
                        status: this.status,
                        statusText: this.statusText
                    });
                }
            },
            onprogress: function(event) {
                console.log(`Loaded ${event.loaded} of ${event.total}`);
            },
            onerror: function(data) {
                reject({
                    status: this.status,
                    statusText: this.statusText
                });
            }
        });
    });
}
  1. Upon returning a successful status within the GM_xmlhttpRequest - the promise returns the JSON as the resolve response.
    function makeRequestWrapper(requestMethod, navToURL, successFunction) {
        makeRequest(requestMethod, navToURL)
            .then(
                function (datums) {
                    successFunction(datums);
                }
            )
            .catch(
                function (err) {
                    console.error('there was an error!', err.statusText);
                }
            );
    }
  1. Have the initial promise, upon success resolve, run a post processing function to update the current window page with the retrieved JSON data
// one of the 3 example handler methods for dealing with the success + JSON
    function successHandler1(jsonDataResponse) {
        let response = JSON.parse(jsonDataResponse);
        let id = response[0].id;
        let updateLink = document.getElementById("unique_link_id").parentNode.parentNode;
        updateLink .innerHTML += ` ID: ${id} `;
    }

Up until this point, my code works as I've intended, the Promise on success returns the JSON data and I update my current window page with that retrieved data.

However, my issue is that some users of the script that pulls this JSON data will have the Promise fail (some more frequently than others) - thus my next step of triage I'm attempting is retrying Promise's a certain number of times, and, with a delay. This way, I figure retrying a failed Promise, let's say 3 times, with 3 second delays, should give ample attempts and time to increase the success rate for the Promise.

My thoughts

Since I am a relative newbie to JS (still learning it and hopefully becomes a language I can use in life), I've read things like Promises and Observables? and rethink if I'm on the right path with this problem. I think I need an in-depth example, specific to my use case, and, with thorough step-by-step explanation, in order to learn more about not only Promises, but also this use-case scenario.

So far, I have attempted to integrate my working code with these SO answers about promises with retry, like:

As well as reading blog posts, like;

I've also looked at countless doc's, like;

And finally, some random (not as well structured) links;

Question

How do I go about converting my existing code to an aforementioned example, or, creating a new API-like method to handle my existing code (work around what I have)?

Nick Bell
  • 516
  • 3
  • 16
  • 1
    You definitely investigated your problem well, and there seems to be a lot of work involved in creating your question, but for me, it seems a bit overly broad. So your question is how to retry a download in case a connection failed? From which point in time would you like to retry? – Icepickle Sep 04 '19 at 16:28
  • @Icepickle - yes, that is my question. Apologies for the setup info, I wanted to be thorough in asking my question. – Nick Bell Sep 04 '19 at 16:31
  • 1
    Boy, I really dislike code that wraps a plain callback into a promise and then makes an interface using that promise that requires the caller to use a plain callback. Just return your promise from your interface please and have the caller use your promise. So .... much more useful. – jfriend00 Sep 04 '19 at 17:04
  • "*I have attempted to integrate my working code with these SO answers*" - please show us the code of these attempts, and describe how they didn't work out. – Bergi Sep 04 '19 at 17:09
  • @jfriend00 could you explain a bit more? I think I understand as, the GM_xmlhttpRequest is wrapped into the Promise - but I don't follow afterwards with what you say; do you mean remove the promise from around the GM_xmlhttpRequest entirely? – Nick Bell Sep 04 '19 at 17:12
  • 1
    `makeRequest()` returns a nice clean promise. Then `makeRequestWrapper()` takes that promise and turns it back into a callback interface. That's what I object to. These days, your caller can use a promise interface just fine and, in fact, can do a lot more with it than they can a callback interface (the whole reason promises where invented). – jfriend00 Sep 04 '19 at 17:15
  • @Bergi The listed bullet points of SO answers = all I have tried. I took the questions and answers from each link, attempted to integrate those answers with my current working code, however I've met an array of errors. the most common being, makeRequestWrapper is not a function when I try to place it into timeout/retry functions. However I've also gotten undefined errors from example answer snippets that don't contain full working examples, and, "refused to connect to undefined blocked by connect CORS check" etc. – Nick Bell Sep 04 '19 at 17:17
  • @NickBell Please show us the most promising of these attempts, and post the array of errors you got. – Bergi Sep 04 '19 at 17:25
  • @Bergi thank you for the help, however I don't think I can fully describe my attempts with the detail I would want to go in, within these limited character comments. I appreciate the incentive to learn though. – Nick Bell Sep 04 '19 at 17:41
  • No, not within the comments. You can [edit] your question to include it. – Bergi Sep 04 '19 at 18:12

1 Answers1

4

Here's a wrapper around your makeRequest() function that lets you pass in the max retry count and the delay between retries. It returns a promise that resolves if successful or rejects if it still has an error after maxRetries.

function makeRequestRetry(requestMethod, navToURL, maxRetry = 3, retryDelay = 3000) {
    let retryCnt = 0;

    function delay(t) {
        return new Promise(resolve => {
            setTimeout(resolve, t);
        });
    }
    
    function run() {
        return makeRequest(requestMethod, navToURL).catch(function (err) {
            ++retryCnt;
            if (retryCnt > maxRetry) {
                console.error('Max retries exceeded. There was an error!', err.statusText);
                throw err;
            }
            console.error('Retry #' + retryCnt + ' after error', err.statusText);
            // call ourselves again after a short delay to do the retry
            // add to the promise chain so still linked to the originally returned promise
            return delay(retryDelay).then(run);
        });
        
    }
    return run();
}

And, one would call it like this:

makeRquestRetry('GET', someURL, 5, 5000).then(result => {
    console.log(result);
}).catch(err => {
    console.log(err);
});

Or, go with the default values for retry parameters:

makeRquestRetry('GET', someURL).then(result => {
    console.log(result);
}).catch(err => {
    console.log(err);
});
marc_s
  • 732,580
  • 175
  • 1,330
  • 1,459
jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • Thank you for the answer, from my test over 100 links, this appears to succeed each time! I still haven't learned promises entirely, however I think tracing the working example helps me visual and learn what call is happening in the call stack and knowing what needs to be returned for each call. Cheers – Nick Bell Sep 04 '19 at 17:39
  • @NickBell - Perhaps you should find out why requests are failing sometimes and then succeeding on retry and see if you can address the root problem. Are you making too many requests in too short a time to one host? – jfriend00 Sep 04 '19 at 22:12