How to make HTTP request with promises and retries in Node.js

Question

I am trying to write a few wrappers around Node HTTP/S module requests, without using axios. node-fetch or any other 3rd party module.

For example, I want to have functions sendGet, sendPost, sendJSON, sendFile etc. In ideal case, these functions will be implementing core functionmakeRequest, just with different parameters.

I want each wrapper to return a promise, so the caller can do anything with the result.

I also want the wrapper to have an argument, how many times will be request retried in case of failure.

So idea is something like this. So far I am able to make a wrapper and pass promise. But I am unable to add ability to retry on failure. It should be (in ideal scenario), part of makeRequest function, but I was unable to to do so, when combined with promises. Thank you for your ideas


// intended usage of wrapper
sendGet('http://example.com', 3).then().catch()

// wrapper applies makeRequest fnc with various arguments 
const sendGet = (url, retries) => {
    return makeRequest('GET', url, retries)
}

const sendPost = (url, retries) => {
    return makeRequest('POST', url, retries)
}

// core function
const makeRequest = async (method, url, retries = 0) => {
    // returns reject on bad status
    // returns resolve with body on successful request
    return new Promise((resolve, reject) => {

        const options = {/*method, hostname, etc */};

        const request = http.request(options, (response) => {
            let chunks = [];

            if (response.statusCode < 200 || response.statusCode >= 300) {
               return reject('bad status code')
            }

            // collect data
            response.on('data', (chunk => {
                chunks.push(chunk)
            }))

            // resolve on end of request
            response.on('end', () => {
                let body = Buffer.concat(chunks).toString();
                return resolve(body)
            })
        })

        request.end();
    })
}

I recently wrote about promise-based `request` in [this Q&A](https://stackoverflow.com/a/67587836/633183) and `retry` in [this Q&A](https://stackoverflow.com/a/67630893/633183). You might also be interested in `Pool` and `throttle` in [this Q&A](https://stackoverflow.com/a/67628301/633183) — Mulan, May 24 '21 at 15:21

carboleda · Answer 1 · 2021-05-24T15:13:44.047

Try this, the original function now is called tryRequest and outside there is the for loop to do the retries

// core function
const makeRequest = async (method, url, retries = 0) => {
    const tryRequest = async () => {
        // returns reject on bad status
        // returns resolve with body on successful request
        return new Promise((resolve, reject) => {

            const options = {method, };

            const request = http.request(url, options, (response) => {
                let chunks = [];

                if (response.statusCode < 200 || response.statusCode >= 300) {
                    return reject('bad status code')
                }

                // collect data
                response.on('data', (chunk => {
                    chunks.push(chunk)
                }))

                // resolve on end of request
                response.on('end', () => {
                    let body = Buffer.concat(chunks).toString();
                    return resolve(body)
                })
            })

            // reject on error of request (Service down)
            request.on('error', function (error) {
                reject(error);
            })

            request.end();
        })
    }

    for (let i=1; i<=retries; i++) {
        try {
            console.log('Try No.', i);
            url = url.substring(0, url.length - 1); // TODO To test, delete after it
            return await tryRequest();
        } catch(error) {
            if (i < retries) continue;
            throw error;
        }
    }
}

Test

await sendGet('http://example.com/abc', 3); // Will work at the 3th try retry
await sendGet('http://example.xyz/abc', 3); // Will fail server not found

Dmitriy Mozgovoy · Answer 2 · 2021-05-24T16:08:46.700

You can use a custom promise & axios or any other promise-based request:

(Live Demo)

import { CPromise } from "c-promise2";
import cpAxios from "cp-axios";

const url =
  "https://run.mocky.io/v3/7b038025-fc5f-4564-90eb-4373f0721822?mocky-delay=2s";

(async()=>{
  const response= await CPromise.retry(() => cpAxios(url).timeout(5000));
})();

More complex:

import { CPromise } from "c-promise2";
import cpAxios from "cp-axios";

const url =
  "https://run.mocky.io/v3/7b038025-fc5f-4564-90eb-4373f0721822?mocky-delay=2s";

const promise = CPromise.retry(
  (attempt) => {
    console.log(`Attempt [${attempt}]`);
    return cpAxios(url).timeout(attempt * 1000 + 500);
  },
  { retries: 3, delay: (attempt) => attempt * 1000 }
).then(
  (response) => console.log(`Response: ${JSON.stringify(response.data)}`),
  (err) => console.warn(`Fail: ${err}`)
);

// promise.pause()
// promise.resume()
// promise.cancel()

How to make HTTP request with promises and retries in Node.js

2 Answers2