28

I read Firestore documentation and all articles on internet(stackoverflow) about Firestore pagination but no luck. I tried to implement the exact code in docs, but nothing happens. I have a basic database with items(over 1250 or more) and I want to get them progressively. By scrolling to load 15 items (to the last item in the database).

If using docs code:

// Construct query for first 25 cities, ordered by population
Query first = db.collection("cities")
    .orderBy("population")
    .limit(25);

first.get()
    .addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
    @Override
    public void onSuccess(QuerySnapshot documentSnapshots) {
        // ...

        // Get the last visible document
        DocumentSnapshot lastVisible = documentSnapshots.getDocuments()
            .get(documentSnapshots.size() -1);

        // Construct a new query starting at this document,
        // get the next 25 cities.
        Query next = db.collection("cities")
            .orderBy("population")
            .startAfter(lastVisible)
            .limit(25);

        // Use the query for pagination
        // ...
    }
});

How to do? Documentation has not too many details.

PS: I need with recycler view (not list view) when user scrolls. Thanks

Johans Bormman
  • 855
  • 2
  • 11
  • 23
  • I think you might be interested in this article, [How to paginate Firestore using Paging 3 on Android?](https://medium.com/firebase-tips-tricks/how-to-paginate-firestore-using-paging-3-on-android-c485acb0a2df). – Alex Mamo Jan 25 '21 at 18:35

3 Answers3

56

As it is mentioned in the official documentation, the key for solving this problem is to use the startAfter() method. So you can paginate queries by combining query cursors with the limit() method. You'll be able to use the last document in a batch as the start of a cursor for the next batch.

To solve this pagination problem, please see my answer from this post, in which I have explained step by step, how you can load data from a Cloud Firestore database in smaller chunks and display it in a ListView on button click.

Solution:

To get the data from your Firestore database and display it in smaller chunks in a RecyclerView, please follow the steps below.

Let's take the above example in which I have used products. You can use products, cities or whatever you want. The principles are the same. Assuming that you want to load more products when user scrolls, I'll use RecyclerView.OnScrollListener.

Let's define first the RecyclerView, set the layout manager to LinearLayoutManager and create a list. We also instantiate the adapter using the empty list and set the adapter to our RecyclerView:

RecyclerView recyclerView = findViewById(R.id.recycler_view);
recyclerView.setLayoutManager(new LinearLayoutManager(this));
List<ProductModel> list = new ArrayList<>();
ProductAdapter productAdapter = new ProductAdapter(list);
recyclerView.setAdapter(productAdapter);

Let's assume we have a database structure that looks like this:

Firestore-root
   |
   --- products (collection)
         |
         --- productId (document)
                |
                --- productName: "Product Name"

And a model class that looks like this:

public class ProductModel {
    private String productName;

    public ProductModel() {}

    public ProductModel(String productName) {this.productName = productName;}

    public String getProductName() {return productName;}
}

This how the adapter class should look like:

private class ProductAdapter extends RecyclerView.Adapter<ProductViewHolder> {
    private List<ProductModel> list;

    ProductAdapter(List<ProductModel> list) {
        this.list = list;
    }

    @NonNull
    @Override
    public ProductViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
        View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.item_product, parent, false);
        return new ProductViewHolder(view);
    }

    @Override
    public void onBindViewHolder(@NonNull ProductViewHolder productViewHolder, int position) {
        String productName = list.get(position).getProductName();
        productViewHolder.setProductName(productName);
    }

    @Override
    public int getItemCount() {
        return list.size();
    }
}

The item_product layout contains only one view, a TextView.

<TextView
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:id="@+id/text_view"
    android:textSize="25sp"/>

And this is how the holder class should look like:

private class ProductViewHolder extends RecyclerView.ViewHolder {
    private View view;

    ProductViewHolder(View itemView) {
        super(itemView);
        view = itemView;
    }

    void setProductName(String productName) {
        TextView textView = view.findViewById(R.id.text_view);
        textView.setText(productName);
    }
}

Now, let's define a limit as a global variable and set it to 15.

private int limit = 15;

Let's define now the query using this limit:

FirebaseFirestore rootRef = FirebaseFirestore.getInstance();
CollectionReference productsRef = rootRef.collection("products");
Query query = productsRef.orderBy("productName", Query.Direction.ASCENDING).limit(limit);

Here is the code that also does the magic in your case:

query.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
    @Override
    public void onComplete(@NonNull Task<QuerySnapshot> task) {
        if (task.isSuccessful()) {
            for (DocumentSnapshot document : task.getResult()) {
                ProductModel productModel = document.toObject(ProductModel.class);
                list.add(productModel);
            }
            productAdapter.notifyDataSetChanged();
            lastVisible = task.getResult().getDocuments().get(task.getResult().size() - 1);

            RecyclerView.OnScrollListener onScrollListener = new RecyclerView.OnScrollListener() {
                @Override
                public void onScrollStateChanged(RecyclerView recyclerView, int newState) {
                    super.onScrollStateChanged(recyclerView, newState);
                    if (newState == AbsListView.OnScrollListener.SCROLL_STATE_TOUCH_SCROLL) {
                        isScrolling = true;
                    }
                }

                @Override
                public void onScrolled(RecyclerView recyclerView, int dx, int dy) {
                    super.onScrolled(recyclerView, dx, dy);

                    LinearLayoutManager linearLayoutManager = ((LinearLayoutManager) recyclerView.getLayoutManager());
                    int firstVisibleItemPosition = linearLayoutManager.findFirstVisibleItemPosition();
                    int visibleItemCount = linearLayoutManager.getChildCount();
                    int totalItemCount = linearLayoutManager.getItemCount();

                    if (isScrolling && (firstVisibleItemPosition + visibleItemCount == totalItemCount) && !isLastItemReached) {
                        isScrolling = false;
                        Query nextQuery = productsRef.orderBy("productName", Query.Direction.ASCENDING).startAfter(lastVisible).limit(limit);
                        nextQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
                            @Override
                            public void onComplete(@NonNull Task<QuerySnapshot> t) {
                                if (t.isSuccessful()) {
                                    for (DocumentSnapshot d : t.getResult()) {
                                        ProductModel productModel = d.toObject(ProductModel.class);
                                        list.add(productModel);
                                    }
                                    productAdapter.notifyDataSetChanged();
                                    lastVisible = t.getResult().getDocuments().get(t.getResult().size() - 1);

                                    if (t.getResult().size() < limit) {
                                        isLastItemReached = true;
                                    }
                                }
                            }
                        });
                    }
                }
            };
            recyclerView.addOnScrollListener(onScrollListener);
        }
    }
});

In which lastVisible is a DocumentSnapshot object which represents the last visible item from the query. In this case, every 15'th one and it is declared as a global variable:

private DocumentSnapshot lastVisible;

And isScrolling and isLastItemReached are also global variables and are declared as:

private boolean isScrolling = false;
private boolean isLastItemReached = false;

If you want to get data in realtime, then instead of using a get() call you need to use addSnapshotListener() as explained in the official documentation regarding listening to multiple documents in a collection. More information you can find the following article:

Alex Mamo
  • 130,605
  • 17
  • 163
  • 193
  • I see your post but is with list view and i want with recycler view and scroll. Can you help me with that? I edited my post. – Johans Bormman Jun 07 '18 at 13:26
  • 1
    As I said, the records are displayed in a `ListView` but that was just an example. You can achieve the same thing using a `RecyclerView`. Have you even try it? – Alex Mamo Jun 07 '18 at 13:33
  • I have tried many examples, docs and i'm stuck for days. Help me please. You have many good posts. – Johans Bormman Jun 07 '18 at 13:45
  • Ok, I'll adapt you the code from there but instead of using a `ListView`, I'll use a `RecyclerView`. – Alex Mamo Jun 07 '18 at 14:02
  • For some reason, I think you should use the new Android Paging Library: https://developer.android.com/topic/libraries/architecture/paging/ as it does not care whether you are using the `RecyclerView/ListView`. I would give it a try, as it seems to be the future as far as pagination in Android is concerned. @JohansBormman – Otieno Rowland Jun 07 '18 at 15:24
  • @RowlandMtetezi I'm trying now to implemt this code. Do you have an example for that? – Johans Bormman Jun 07 '18 at 15:33
  • @RowlandMtetezi You're right but I saw in the link you provided, the documentation is only for Kotlin. The OP asked for Android. Thanks! – Alex Mamo Jun 07 '18 at 15:35
  • The example I have is Paging with a network sample: https://github.com/googlesamples/android-architecture-components/tree/master/PagingWithNetworkSample , I have yet to see ane example working with firebase or firestore. However, I think you can easily adapt/borrow the concept. – Otieno Rowland Jun 07 '18 at 15:44
  • @AlexMamo does not Kotlin write Android apps as early as 2016? Kotlin is first class, just like Java. – Otieno Rowland Jun 07 '18 at 16:12
  • @RowlandMtetezi Ya, you're right. I haven't seen the Kotlin tag in OP's question, so I wrote the code in Android. Thanks! – Alex Mamo Jun 07 '18 at 16:24
  • @RowlandMtetezi Thanks for the informations. I don't know kotlin but as soon as I learn, I give it a try. I'll stay for the moment with Java. – Johans Bormman Jun 07 '18 at 17:55
  • 1
    What to say Alex, it took me 3 hours but I finally made it. Thank you by the way! – Johans Bormman Jun 07 '18 at 17:57
  • I am trying to add it from many hours but isscrolling is always false even after scrolling in real device and onScrollStateChanged is not calling at all , please help – gautam Aug 17 '18 at 18:46
  • Found the problem it is with nested scrolling, anyone know how to fix it? – gautam Aug 17 '18 at 19:10
  • @AlexMamo plz have a look at https://stackoverflow.com/questions/52021119/save-documentsnapshot-for-pagination-in-firestore-android – Zar E Ahmer Aug 27 '18 at 10:47
  • I converted same code in kotlin, and run then first query works properly second time app crashed with java.lang.RuntimeException: No properties to serialize found on class com.google.firebase.firestore.A this error – Yogesh Rathi Jul 08 '19 at 12:14
  • @YogeshRathi Please post another fresh question regarding this issue, so me and other Firebase developers can help you. – Alex Mamo Jul 08 '19 at 12:15
  • @AlexMamo please look into this issue https://stackoverflow.com/q/56964430/5096868 – Yogesh Rathi Jul 10 '19 at 06:19
  • how to get post id means document id i put limit 3, so task give me only 3 documents id, but after 4th document i got error, which is i have only 3 documents , please help me about this, your post help me too much, thank you – Qutbuddin Bohra Feb 09 '20 at 07:06
  • @QutbuddinBohra Without seeing your code I cannot be much of a help. So please post another fresh question, so I and other Firebase developers can help you. – Alex Mamo Feb 10 '20 at 09:08
  • I posted new questions regarding how to get document id after next query run. link https://stackoverflow.com/questions/60247307/how-to-get-documents-id-firestore-paginate?noredirect=1#comment106587412_60247307 please help me about this i also tried firebase pagging adapter ui but its doesnt show me live data. – Qutbuddin Bohra Feb 23 '20 at 10:20
  • @AlexMamo on your Real time data load work, after notifyDataSetChanged() on adapter its refresh whole item which under LIMIT. Help about how to not refresh things under its item, only refresh that item which is needed. – Qutbuddin Bohra Mar 08 '20 at 06:03
  • @QutbuddinBohra If you are interested in a Firestore real-time pagination algorithm you can take a look this [example](https://github.com/alexmamo/FirestoreRealtimePagination), which is a repo for this [article](https://medium.com/firebase-tips-tricks/how-to-create-a-clean-firestore-pagination-with-real-time-updates-ce05a87bb902). – Alex Mamo Mar 08 '20 at 09:03
  • Still looking fore firestore pagination, check this link https://techtibet.com/blog/android/how-to-paginate-firestore-with-android/ – Kunchok Tashi Mar 31 '20 at 12:48
  • @kontashi35 I'm sorry but I cannot see any pagination at all in that post. – Alex Mamo Mar 31 '20 at 12:57
  • @AlexMamo Thanks for that, file was missing.. added now – Kunchok Tashi Mar 31 '20 at 14:04
  • 1
    @kontashi35 I cannot see the reason why you added that article as it's exactly like my answer :| – Alex Mamo Mar 31 '20 at 14:15
  • @AlexMamo they are people who prefer code over explanation ,demo over theory and also the way we explain. I just thought it will be usedful to some one(even a person) .I will remove it if you found it not good. – Kunchok Tashi Mar 31 '20 at 14:39
  • For some reason, I am always getting empty result when using snapshot in `startAfter`. Is there a catch? I have a simple order as explained above. If I use the field value for the cursor it works, but when I put there a `DocumentSnapshot` it succeeds but the result documents are empty – parohy Jun 25 '20 at 15:47
  • @parohy Without seeing your code, I cannot be much of a help. So please post a new question using its own [MCVE](https://stackoverflow.com/help/mcve), so I and other Firebase developers can help you. – Alex Mamo Jun 25 '20 at 16:59
  • @AlexMamo could please take a look at this https://stackoverflow.com/questions/63295921/how-to-convert-a-map-into-arraylist-from-firestore-document/63296206?noredirect=1#comment111928392_63296206 . – AndroidRocket Aug 07 '20 at 10:34
  • This approach is not good. While you get updates in real time, realise that when paginating the query you create a new query when fetching the next set of documents. You therefore reassign the old query handle to the new query. Removing the old event listeners is not possible when the lifecycle of the component(activity, fragment...) is done. We know that listeners should be removed when not needed to avoid unnecessary costs: bandwidth, battery and sever reads. – thinclient Jan 04 '22 at 05:33
  • @thinclient This approach is not for getting updates in real-time. As you can see, it uses `get()`, which gets the data **only** once. – Alex Mamo Jan 04 '22 at 08:51
  • @AlexMamo sorry I missed the get() part. I'm so used to attaching an event listener. I did read another answer of yours where there is a link to a medium article where you are creating a view model and attaching event listeners and actually getting real time updates. I will comment there. – thinclient Jan 04 '22 at 09:58
8

FirebaseUI-Android also recently came out with a Firestore Paginator.

I have used it in my code, and it works great - just keep in mind that it operates using .get() instead of .addSnapshotListener(), so the recycler is not in realtime.

See the docs here:

https://github.com/firebase/FirebaseUI-Android/tree/master/firestore#using-the-firestorepagingadapter

Jeff Padgett
  • 2,380
  • 22
  • 34
  • The FirebaseUI-Android component is quick and easy to get up and running. If you need to do more advanced queries or filtering that Firebase does not allow you can create your own **DataSource** with a **PagedListAdapter** as I outlined in my answer here. https://stackoverflow.com/questions/51859652/how-to-exclude-an-element-from-a-firestore-query/52048159#52048159 – AdamHurwitz Aug 27 '18 at 23:33
  • 1
    I tried using this, and its probably the most quick way of doing it. But Pagination with FirebaseUI doesn't seem like the right way if you have +1000 plus documents. It loads all the documents at once with a .get() query and then paginates it locally. Any idea how can we make this more dynamic? fetching only 10-20 records at once? – Siddhesh Dighe Dec 22 '19 at 11:27
  • @SiddheshDighe Why do you think it loads all the documents at once? I think it only loads one page at a time, and you set the size of the page. See: https://github.com/firebase/FirebaseUI-Android/blob/master/firestore/src/main/java/com/firebase/ui/firestore/paging/FirestoreDataSource.java – Jeff Padgett Dec 24 '19 at 15:58
  • He need load the all documents or not? Because if he loads all documents it's not work fine... – luke cross May 18 '20 at 13:27
  • Im using FirestorePagingAdapter from FirebaseUI , and wrote a query which include whereEqualTo() but it doesnt seem to be working . The return data is not sorted even if I used whereEqualTo() is it receiving the complete dataset from firestore – JustGotStared Feb 10 '22 at 18:27
1

You can also use FirestorePagingAdapter provided by Firebase-UI-Firestore

You need to install this dependency

 implementation 'com.firebaseui:firebase-ui-firestore:latest_version_here'

Solution

Step 1: Create a global Firestore paging adapter variable and pass the Model class and ViewHolder, and also the Model variable.

private FirestorePagingAdapter<Model, ModelViewHolder> adapter;
private Model model;

Step 2: Create a firebase query

Query query = db.collection("cities")
    .orderBy("population");

Step 3: Let's build the pagedlist config. Here you will pass how much data to be queried in each page;

PagedList.Config config = new PagedList.Config.Builder()
                .setEnablePlaceholders(false)
                .setPrefetchDistance(10)
                .setPageSize(15)
                .build();

Step 4: After setting the config, let's now build the Firestore paging options where you will pass the query and config.

 FirestorePagingOptions<Model> options = new FirestorePagingOptions.Builder<Model>()
                .setLifecycleOwner(this)
                .setQuery(query, config, snapshot -> {
                    model = snapshot.toObject(Model.class);
                    return model;
                })
                .build();

Step: 5 Now let's pass the data to the Recylerview

 adapter = new FirestorePagingAdapter<Model, ModelViewHolder>(options) {
            @Override
            protected void onBindViewHolder(@NonNull ModelViewHolder holder, int position, @NonNull Model model) {
                holder.bindTO(model);
            }

            @NonNull
            @Override
            public ModelViewHolder onCreateViewHolder(@NonNull ViewGroup parent, int viewType) {
                View view = LayoutInflater.from(parent.getContext()).inflate(R.layout.layout_model, parent, false);
                return new ModelViewHolder(view);
            }

            @Override
            protected void onError(@NonNull Exception e) {
                super.onError(e);
                     //logic here
            }

            @Override
            protected void onLoadingStateChanged(@NonNull LoadingState state) {
                switch (state) {
                    case LOADING_INITIAL:
                        break;

                    case LOADING_MORE:
                        break;

                    case LOADED:
                        notifyDataSetChanged();
                        break;

                    case ERROR:
                        Toast.makeText(requireActivity(), "Error", Toast.LENGTH_SHORT).show();
                         //logic here
                        break;

                    case FINISHED:
                        //logic here
                        break;
                }
            }
        };
        productRecycler.setAdapter(adapter);
        adapter.notifyDataSetChanged();
    }

Happy Coding!

Denny
  • 991
  • 12
  • 20