Asynchronous access to local variables in Node.js

Question

In my node.js backend I have this global variable, which is a key-value set.

responses = {};

Entries are continuously created by an asynchronous task

function responseContinuousFetching(){
    axios.get(url, 
    ).then((response) => {
       responses[response.guid] = response.payload;
       setTimeout(()=>responseContinuousFetching(), 1);
    }).catch(error) {
       [handle errors]
       setTimeout(()=>responseContinuousFetching(), 1);
    };
};

Another asynchrounous task checks this key-value set. When the right id is found the entry is removed from the set

api.post(url, jsonParser, (req.res) => {
   const params = req.body;
   [some synchronous code]
   let found = false;
   while (!found) {
      if responses.hasOwnProperty(req.body.guid) {
         response = responses(req.body.guid);
         delete responses[req.body.guid];
         found = true;
      }
    }
    return response;
});

(For clarity: the "responseContinuousFetching" is done by the backend to an external API, while the second block is the definition of an entrypoint, which is exposed by the backend to a frontend)

This obviously doesn't work, as the while loop is synchronous code which monopolizes the backend, preventing the update of the object "responses". The result is that the while loop goes on forever and the backend is stuck.

I was wondering if there is any way to access asynchronously to local variables, allowing other tasks that are waiting in the event loop to be executed.

Thanks,

Thomas

Yes. I simplified things in the examples, but the guid is created each time randomly with uuidv4(), thus it is unique. — zeroquaranta, Jun 13 '22 at 17:01

score 1 · Answer 1 · answered Jun 13 '22 at 16:58

You need a promise that resolves when the entry is found.

You could store a list of temporary resolvers if the entry isn't yet found, and have responseContinuousFetching call them on a successful response.

For example:

const responses = {};
const resolvers = {};

function getResponseByGuid(guid) {
  return new Promise((resolve) => {
    if (responses.hasOwnProperty(guid)) {
      const response = responses[guid];
      delete responses[guid];
      resolve(response);
    } else {
      resolvers[guid] = resolve;
    }
  });
}

function responseContinuousFetching() {
  axios
    .get(url)
    .then((response) => {
      if (resolvers[response.guid]) {
        const resolver = resolvers[response.guid];
        delete resolvers[response.guid];
        resolver(response.payload);
      } else {
        responses[response.guid] = response.payload;
      }
    })
    .catch((error) => {
      // handle error
    })
    .finally(responseContinuousFetching);
};

I would implement some kind of time-based eviction policy on responses and resolvers objects though. For example, what if a guid is never requested? Will it just stay forever in that responses object? How many can there be?

this way will "getResponseByGuid" perform continuous check on "responses"? or just once? — zeroquaranta, Jun 13 '22 at 17:16
No, `getResponseByGuid(req.body.guid)` is intended to be called by your `api.post(...)` endpoint. Then, you respond only once the returned promise has resolved. — sp00m, Jun 13 '22 at 17:18
the thing is at the moment I'm calling `getResponseByGuid(req.body.guid)` I don't know when the response will be available in `responses`...that's why somehow I need to keep watching `responses`.. sorry but I don't understand the use of `resolvers`. More sepcifically I don't understand this `... else { resolvers[guid] = resolve; } ...` — zeroquaranta, Jun 13 '22 at 17:25

zeroquaranta · Accepted Answer · 2022-06-14T15:06:45.213

Considering suggestions by sp00m and this thread I ended up with this solution:

const response; 
function responseContinuousFetching() {
   axios
      .get(urlForNewResponses)
      .then((response) => {
         responses[response.guid] = response.payload;
      })
      .catch((error) => {
          // handle error
      })
      .finally(responseContinuousFetching);
};
[...]
api.post(url, jsonParser, (req.res) => {
   const params = req.body;
   [some synchronous code]
   const guid = uuid(v4);
   const response;
   async function waitForResponse() {
      return new Promise((resolve) => {
         function getResponse() {
            if (responses.hasOwnProperty(guid)) {
               response = responses[guid];
               delete responses[guid];
               resolve();
            }
            else  {            
               setTimeout(getResponse, 1); 
            }
         }
         getResponse();
      });
   }
   await waitForResponse();
   return response;
});

What I get is I'm able to iteratively poll a local data structure, until I find what I'm searching, without blocking the entire back-end. Still this check is done "synchronously" by using await. This way I can properly serve the required value in the endpoint implementation.

The data structure is fed by another task, which continuously fetches data from an external source.

Asynchronous access to local variables in Node.js

2 Answers2