How can we paginate a firebase / firestore query on ios swift?

Question

I try to find a solution for paginate a firebase query on ios/swift but I couldn't build algorithm for my state.

My method is like this:

func downloadData(completion: @escaping ([Post]) -> Void) {
        
        // download data with pagination
        
        let firestoreDatabase = Firestore.firestore()
        
        var first =  firestoreDatabase.collection("posts").order(by: "date", descending: true).limit(to: 5)
        
        first.addSnapshotListener{ snapshot, error in
            
            guard let snapshot = snapshot else {
                print("Error retrieving cities: \(error.debugDescription)")
                return
            }
            
            guard let lastSnapshot = snapshot.documents.last else {
                // The collection is empty.
                return
            }
            
            self.postList.removeAll(keepingCapacity: false)
            
            DispatchQueue.global().async {
                
                for document in snapshot.documents {
                    
                    // getting data from document stuff ...
                    
                    self.postList.append(self.post)
                }
                
                completion(self.postList)
            }
            
            // how can I repeat this query as long as lastSnapshot exist
            firestoreDatabase.collection("posts").order(by: "date", descending: true).start(afterDocument: lastSnapshot).addSnapshotListener { querySnapshot, error in
            }
        }
}

I tried following mindset but it didn't work, and entered an infinite loop. I didn't understand why it is.

func downloadData(completion: @escaping ([Post]) -> Void) {
        
        // download data with pagination
        
        let firestoreDatabase = Firestore.firestore()
        
        var first =  firestoreDatabase.collection("posts").order(by: "date", descending: true).limit(to: 5)
        
        first.addSnapshotListener{ snapshot, error in
            
            guard let snapshot = snapshot else {
                print("Error retrieving cities: \(error.debugDescription)")
                return
            }
            
            guard let lastSnapshot = snapshot.documents.last else {
                // The collection is empty.
                return
            }
            
            self.postList.removeAll(keepingCapacity: false)
            
            DispatchQueue.global().async {
                
                for document in snapshot.documents {
                    
                    // geting data from document stuff ...
                    
                    self.postList.append(self.post)
                }
                
                completion(self.postList)
            }

            repeat {
                firestoreDatabase.collection("posts").order(by: "date", descending: true).start(afterDocument: lastSnapshot).addSnapshotListener { querySnapshot, error in
                    
                    guard let snapshot = snapshot else {
                        print("Error retrieving cities: \(error.debugDescription)")
                        return
                    }
                    
                    guard let lastSnapshot = snapshot.documents.last else {
                        // The collection is empty.
                        return
                    }
                    
                    self.postList.removeAll(keepingCapacity: false)
                    
                    DispatchQueue.global().async {
                        
                        for document in snapshot.documents {
                            
                            // getting data from document stuff ...
                            
                            self.postList.append(self.post)
                        }
                        
                        completion(self.postList)
                    }
                    
                    lastSnapshot = snapshot.documents.last
                }
            } while(lastSnapshot.exists)
        }
}

I think lastSnapshot must be nil after the query loop but it is appear that it is still exist.

how can I fix lastSnapshot problem? Or is there different mindset / easiest way to paginate?

In firebase documents, it says just use this but how can we repeat query that has " .start(afterDocument: lastSnapshot) " stuff?

The problem with your second approach is that your loop is outside of the asynchronous completion block. You can't use a loop. In your completion handler block you can recursively call 'downloadData`, passing the last snapshot. If you have a large number of records you may not want to fetch them all at once. Depending on how you are displaying them it might be better to fetch the next records just before you need to display them (say as a table view scrolls) — Paulw11, Dec 22 '22 at 21:07
thank you so much mr marc_s I will immediately try when I wake up. this was crucial affair for my app, since I will finish it soon. thank you again, I will edit this comment after try — hyasar, Dec 22 '22 at 22:07

trndjc · Answer 1 · 2023-03-06T18:01:22.320

0

First and foremost, for plain-vanilla pagination, don't use a snapshot listener when fetching documents. You can paginate documents with a snapshot listener but the process is more complex.

I've embedded my notes into the comments in the code below for clarity.

let pageSize = 5
var cursor: DocumentSnapshot?

func getFirstPage(completion: @escaping (_ posts: [Post]?) -> Void) {
    let db = Firestore.firestore()
    let firstPage = db.collection("posts").order(by: "date", descending: true).limit(to: pageSize)
    
    firstPage.getDocuments { snapshot, error in
        guard let snapshot = snapshot else {
            
            // Don't leave the caller hanging on errors; return nil,
            // return a Result, throw an error, do something.
            completion(nil)
            
            if let error = error {
                print(error)
            }
            return
        }
        guard !snapshot.isEmpty else {
            
            // There are no results and so there can be no more
            // results to paginate; nil the cursor.
            cursor = nil
            
            // And don't leave the caller hanging, even without
            // results; return an empty array.
            completion([])
            
            return
        }
        
        // Before parsing the snapshot, manage the cursor.
        if snapshot.count < pageSize {
            
            // This snapshot is smaller than a page size and so
            // there can be no more results to paginate; nil
            // the cursor.
            cursor = nil
        } else {
            
            // This snapshot is a full page size and so there
            // could potentially be more results to paginate;
            // set the cursor.
            cursor = snapshot.documents.last
        }
        
        var posts: [Post] = []
        
        for doc in snapshot.documents {
            posts.append(newPost) // pseudo code
        }
        completion(posts)
    }
}

func continuePages(completion: @escaping (_ posts: [Post]?) -> Void) {
    guard let cursor = cursor else {
        return
    }
    let db = Firestore.firestore()
    let nextPage = db.collection("posts").order(by: "date", descending: true).limit(to: pageSize).start(afterDocument: cursor)
    
    nextPage.getDocuments { snapshot, error in
        guard let snapshot = snapshot else {
            completion(nil)
            if let error = error {
                print(error)
            }
            return
        }
        guard !snapshot.isEmpty else {
            
            // There are no results and so there can be no more
            // results to paginate; nil the cursor.
            cursor = nil
            
            completion([])
            return
        }
        
        // Before parsing the snapshot, manage the cursor.
        if snapshot.count < pageSize {
            
            // This snapshot is smaller than a page size and so
            // there can be no more results to paginate; nil
            // the cursor.
            cursor = nil
        } else {
            
            // This snapshot is a full page size and so there
            // could potentially be more results to paginate;
            // set the cursor.
            cursor = snapshot.documents.last
        }
        
        var morePosts: [Post] = []
        
        for doc in snapshot.documents {
            morePosts.append(newPost) // pseudo code
        }
        completion(morePosts)
    }
}

edited Mar 06 '23 at 18:01

answered Dec 23 '22 at 02:57

trndjc

11,654
3
38
51

Thank you so much for answer. I tried but I didn't run these functions together. How can I run two of these func as long as that lastSnapshot is exist? should I use loop? I tried loop but error occured about sync and threads. – hyasar Dec 23 '22 at 15:36
@hyasar I don’t understand why you’d want to run them in a loop. The purpose of pagination is to load data in blocks. Running them in a loop would effectively make it one giant data grab. What are you trying to ultimately achieve here? – trndjc Dec 23 '22 at 15:38
these funcs are in WebService folder in my app , and I call these func from another folder. I couldn't handle it. Isn't it suppose to run these func as long as lastSnapshot is exist? When I call these two funcs each of them just running one time. for instance, I have 20 posts in database but 10 posts just coming. thing that I didn't understand is how can I handle recursive thing – hyasar Dec 23 '22 at 15:46
@hyasar what are you ultimately trying to achieve here from the end user’s perspective? What is the purpose of the snapshot listener and the loop? – trndjc Dec 23 '22 at 15:57
In normal when I slide my finger down on the screen the new posts should come from database. Isn' it right? So I couldn't handle it. This is what I'm saying. how are two of funcs going to be run for this? these are seperate funcs. how are they going to know know each other? which one will I call? – hyasar Dec 23 '22 at 16:00
when I call these func by sequential, just 10 posts coming from db the others are missing although slide the simulator screen. they never come. – hyasar Dec 23 '22 at 16:11
@hyasar they don’t need to know each other. Call the first function when the screen loads and then call the other function whenever the user scrolls to the bottom of the screen. Both functions read and write to the same cursor variable, that is the key to making this work. The cursor variable is the only bit of information both functions need to share, except for the query itself which you can hardcode. – trndjc Dec 23 '22 at 16:12
okey I really understand now. thank you so much for your attention and answers. now I need to learn how can detect whenever user scrolls to the bottom of tableView. thank you again, I guess I can handle it rest of algorithm. have a good day mr trndjc – hyasar Dec 23 '22 at 16:25
Ok but what if I need querysnapshot AND also pagination? How does it work then? The user needs to browse older data via pages, but also need to be notified of new data at the front of the list. Now what? – erotsppa Mar 06 '23 at 17:58
@erotsppa the problem with the snapshot listener is that it only listens for changes to the query itself, which for paginated results is only the first page of results. If there is a change to a document on any other page, the snapshot listener won't notice it and the results won't change. You can create a ghost document that's always on page 1 that's always changed when any document is changed. But if your results are sorted by something like a timestamp and changing any document will put it at the top of the query then you don't need this ghost document. – trndjc Mar 06 '23 at 18:05
@erotsppa once you've figured this part out then all that remains is keeping track of how many pages the user currently has in view and reloading that many when the first page of the query changes. – trndjc Mar 06 '23 at 18:07
@trndjc I'm currently implementing a simple list that needs to be in sync with firestore. It's seemingly impossible as outlined in detailed here https://stackoverflow.com/questions/75647101/what-is-the-correct-with-to-do-pagination-with-firebase-firestores-querysnapsho essentially the gist of it is that (1) I need a listener to track the changes coming at the top of list (sorted by timestamps) so that as new items are added I am notified but (2) I also need pagination so that when the user reaches the end of list, I need to fetch the next page but the original querySnapshot cannot do that – erotsppa Mar 06 '23 at 20:31
@erotsppa it is definitely doable and relatively straightforward. I have it in my app. When the list changes, in any way, will it always change the results of the first page? – trndjc Mar 06 '23 at 21:09
@trndjc no the first page is driven by new addition to the list, the old results are driven by pagination controls only. Is there a way to chat with you? – erotsppa Mar 06 '23 at 22:56
@erotsppa if we keep commenting back and forth, eventually a chat room will be created for us automatically. Is the snapshot listener listening for new documents + changes to existing documents then? – trndjc Mar 06 '23 at 23:07
Let's say I have 100 posts in a facebook feed list. I have to create a listener say for the first 20. Now whenever someone adds to this feed, I get notified by the listener. Ok. But now the user scroll down the list and I have to get #20-40. How do I do that? Another listener? So now I will have 5 listeners, not to mention the need to merge the 5 returned results. Ok another approach. Let's say I just call a fetch for the next 20. Ok I got the results, now what do I do? Do I merge it with the first 20 from the listener? It's such complicated code. Why can't firebase just sync the damn database – erotsppa Mar 06 '23 at 23:30
Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/252334/discussion-between-trndjc-and-erotsppa). – trndjc Mar 06 '23 at 23:41
@erotsppa chatroom made – trndjc Mar 07 '23 at 00:28

How can we paginate a firebase / firestore query on ios swift?

1 Answers1