Express middleware with async operation that happens only once

Question

I am trying to append some data to my request object using a middleware, but I want to do it only once the server is up. So I tried doing it with a middleware, while trying to use a function's context, but it's a bit problematic to perform such an action on a middleware, because I cannot pass a promise as a middleware.

This is what I'm trying to do:

const setupData = async () => {
    const data = await getSomeData();
    return (req, res, next) => {
        req.data = data;
        next();
    }
}

app.use(setupData());

I tried using the solution suggested here, but it won't work as this will happen on every request.

Any idea how can I go around this? I can always put the info on a global var, but I would like to avoid it. I also saw some in-memory packages to help with it (such as node-cache), but I would like to do it with a middleware.

Thanks in advance

slebetman · Accepted Answer · 2020-10-05T12:01:35.393

Just cache the result using a normal variable:

let data = null;

function setupData (req, res, next)  {
    if (data !== null) {
        req.data = data;
        next();
    }
    else {
        getSomeData().then(result => {
            data = result
            req.data = data;
            next();
        });
    }
}

app.use(setupData);

This is the minimal, least complicated implementation. You can of course refactor it to be much DRYer and less prone to mistakes by taking out the caching logic:

Cleaner Implementation

let cache = null;
async function getCachedData() {
    if (cache === null) {
        cache = await getSomeData();
    }
    return cache;
}

Which makes setupData much cleaner:

function setupData (req, res, next) {
    getCachedData().then(data => {
        req.data = data;
        next();
    });
}

Either way, the cache is triggered on the first request. This means that there is a possibility that a second request may arrive before the data is possibly cached. So at startup the getSomeData() function may run more than once.

Really call getSomeData() ONLY ONCE

If you really want to call getSomeData only once you must call it before setting up Express:

async function main () {

    const data = await getSomeData();

    const app = express();

    //
    // set up express middlewares...
    //

    app.use((req,res,next) => {
        req.data = data;
        next();
    });

    //
    // set up routes...
    //

    app.listen(config.port);
}

main(); // start everything

The key here is to realize that we have been trying to do everything backwards: to set up a constant value asynchronously AFTER starting to set up Express. The natural flow of the program wants the constant value to exist BEFORE we begin setting up Express so we only perform everything else inside the async function (here called main). Not try to run the async function while setting up Express.

Thanks, I wanted to avoid that. I thought that there's another, more elegant way. — matmiz, Oct 05 '20 at 11:59
@matmiz The refactored caching logic is very elegant IMHO. You can stuff it in a module and not see the mechanics of it. It's the same if you use 3rd party libraries that do similar things like jQuery or knex or rxjs or React etc. They all maintain messy states within them but we never need to see it — slebetman, Oct 05 '20 at 12:03
Thanks @slebetman. You edited answer is much clear now, you got a good point there (and also in your comment). Cheers! — matmiz, Oct 05 '20 at 12:04
The caching solution is still very useful to know in case you don't have the luxury of writing the `main()` function. For example if you want to publish a library/module for others to use. Internally this is how libraries like knex or mysql store connections. Notice that most db libraries have what looks like a synchronous API to set up connections. But in actual fact the connection is only created on first request and then get cached - similar to the caching logic above but usually more complicated because they also try to pool connections — slebetman, Oct 05 '20 at 12:10

Deepak Gupta · Answer 2 · 2020-10-05T11:49:18.797

0

You can do it without async:-

const setupData = (req, res, next) => {

// You can put a condition here so that it runs only once
    getSomeData().then((data) => {
    req.app.locals.data = data //The data can be accessed in the next middleware using req.app.locals
    next();
}).catch((error) => {
console.log("Error Occured");
res.status(400).end("Error Occurred");
})
}
app.use(setupData);

You should see the documentation of getSomeData and see how it works

edited Oct 05 '20 at 11:49

answered Oct 05 '20 at 11:17

Deepak Gupta

614
1
7
14

Thanks, but that won't work. You're not really passing the req,res and next to the `setupData` function. Even if you do, that brings be back to the original problem, that I want it to happen only once – matmiz Oct 05 '20 at 11:36
See the updated answer. The `req, res, next` are getting passed in the function. – Deepak Gupta Oct 05 '20 at 11:40
Thanks, but as I mentioned - I want it to happen only *once*, when I load the app. Your solution will fetch the data on every request. – matmiz Oct 05 '20 at 11:43
Put an if else condition so that it wont fetch if the variable is already set. – Deepak Gupta Oct 05 '20 at 11:46
Yep, that's what I had in mind. I wanted to avoid this sort of solution, but it looks like I don't have any other option – matmiz Oct 05 '20 at 11:49

Express middleware with async operation that happens only once

2 Answers2

Cleaner Implementation

Really call getSomeData() ONLY ONCE