3

I want to do pagination with redis cache, i'am using Node.js as my back-end and i'am using npm redis-scanner package for scanning through the keys . Here i'am unable to go to a particular key and fetch next 15 keys from redis cache. How do i do it?

shubham urolagin
  • 75
  • 1
  • 6

1 Answers1

0

You can use ioredis like so:

const Redis = require("ioredis");

async getByCursor({ cursor }) {
    const db = new Redis();
    const stream = db.scanStream({ count: 5 }); // count = number of items to return per page
    try {
      const scan = () =>
        new Promise((resolve, reject) => {
          let items = [];
          stream.on("data", (page) => {
            items = [...items, page]; // page = ['key1', 'key2', etc.]
          });
          stream.on("end", () => {
            resolve(items); // items = [['key1', 'key2'], ['key3', 'key4']]
          });
        });
      const keys = await scan();
      return Promise.all(
        keys[cursor].map(async (key) => JSON.parse(await db.get(key)))
      );
    } catch (e) {
      console.error(e);
      return Promise.resolve([]);
    }
  }

or slightly faster:

async getByCursor({ cursor }) {
    const stream = db.scanStream({ count: 5 });
    try {
      return new Promise((resolve, reject) => {
        let pageCount = 0;
        stream.on("data", (page) => {
          pageCount = pageCount + 1;
          if (pageCount == Number(cursor)) {
            resolve(
              Promise.all(
                page.map(async (id) => unpack(await db.get(id)))
              ).catch((e) => {
                console.error(e);
                reject([]);
              })
            );
          }
        });
      });
    } catch (e) {
      console.error(e);
    }

Using cursors

Here cursor value 1 provided e.g. as param or query when calling your 'items-by-cursor' node route gives you the first 5 items, value of 2 the next page etc. Note that the count is just an estimate, i.e. you can't rely on it always returning strictly 5 keys (see the documentation link below for details). You can check https://react-query.tanstack.com/examples/load-more-infinite-scroll for a frontend solution to obtain the relevant cursors to be sent to the route.

Note on performance

Fetching the key-value pairs via a stream is non-blocking in comparison to getting all items via all keys by calling db.get('*') and/or filtering the result in your route. Therefore this implementation is recommended for production.

Further ioredis documentation

https://github.com/luin/ioredis#streamify-scanning

pspray
  • 34
  • 6