Firestore pagination: how to query last page

Question

I have created an public restful POST api for searching using Firebase function and firestore:

app.post("/", async (req, res) => {
    const { eventIds, queryString, startAtIndex, limit } = req.body;
    let query = firestore()
        .collection("IndexedEcImage_test")
        .where("eventId", 'in', eventIds);

    if (queryString) {
        query = query.where("generatedSearchKeys", "array-contains", queryString);
    }

    query = query.orderBy("timestamp")

    if (startAtIndex) {
        query = query.startAfter(startAtIndex);
    }

    const snapshot = await query.limit(limit).get();
    const results = snapshot.docs.map(doc => doc.data());

    res.status(200).send({
        indexForNextPage: results[results.length - 1].timestamp,
        data: results
    });
});

The body request:

{
    "eventIds": [
        "9m6k89aOlqeXpaJ3oZhW1"
    ],
    "queryString": "ABC",
    "startAtIndex": 1637128385472,
    "limit": 2
}

The output is like this: { "indexForNextPage": 1637128385472, "data": [{...},{....}] }

For example: The "IndexedEcImage_test" collection has 5 docs. I expected that the pagination would be like that:

Page 1: first 2 docs => Page 2: next 2 docs => Last page: the last doc

The indexForNextPage in the output is used to fetch the next page, but when it comes to the last page, it gave me a timeout. I guess the limit goes beyond the number of the remaining docs. The number of remaining docs can change differently depending on the total docs or the limit. Is there a general way to fetch the last page without counting all the docs?

Answer 1

I will start by recommending you read this note and the upgrade guide. In case you might be using an old tutorial and you have accidentally merged the two versions, it's a common mistake that could cause issues. If that is not the case, and you are opting to use the 8 version I found two old questions using that SDK version that might help you. In this one, the advice is...

To detect whether there are more pages to load, you'll need to request an additional item. So, since you seem to show 2 items per page, you have to request 3 items - and display only two of them. If you get a third item, you know there's an additional page.

and the other post contains this piece of code that in my opinion could be useful:

$("#js-previous").on('click', function () {
   $('#employee-table tbody').html('');
   var previous = db.collection("employees")
       .endBefore(firstVisible)
       .limitToLast(3);
   previous.get().then(function (documentSnapshots) {
       documentSnapshots.docs.forEach(doc => {
           renderEmployee(doc);
       });
   });
});

---

However, if you consider using the latest version 9 SDK, once you have made a migration this would be a suggested way of Paginate a query to get the last document in a batch. You can compare this to the 8 SDK version and see the differences on the code they use to get the last element:

var first = db.collection("cities")
        .orderBy("population")
        .limit(25);

return first.get().then((documentSnapshots) => {   // Get the last visible document   
var lastVisible = documentSnapshots.docs[documentSnapshots.docs.length-1];  
console.log("last", lastVisible);

  // Construct a new query starting at this document,   
  // get the next 25 cities.   
var next = db.collection("cities")
          .orderBy("population")
          .startAfter(lastVisible)
          .limit(25); 
 }); 

Note: All the code samples are using version 8 of the sdk

The video in the same page is very informative and can be very useful to understand how pagination works.