1

I have an app where user can post item displayed in RecyclerView where Cloud FireStore is the backend. Now in first launch of the app it will load first 5 items. If RecyclerView cannot scroll vertically anymore it will fetch another 5 items. Everything is working except when I delete one item, other item gets duplicate.

First Problem Scenario:

In first launch it will load 5 items, so my list only have 5 items at this moment. Since my pagination limit is 5 when I delete one item from that 5 items, the query listeners tries to load the 6th item. In that part when I scroll up to load the next 5 items I'll get duplicate 6th item.

Like this 1, 2, 3, 4, 5 then the 3rd item will be deleted 1, 2, 4, 5 should be the result Unfortunately this is what I get 1, 2, 4, 5, 6. Its good to see that the query itself tries to load another 1 item after 1 item is deleted but after scrolling up the RecyclerView it will load another 5 items. Then this is what I get

1, 2, 4, 5, 6, 6, 7, 8, 9, 10

But since every new added item is going displayed at the top which is 0 index so it means that what I really see in my list is 6, 1, 2, 4, 5, 6, 7, 8, 9, 10.

My idea: Do I need to update my DocumentSnapshot lastSeen value too in every delete action or should I adjust the value of limit() dynamically? Please enlighten me of what is the best way to deal with it.

Sample code:

 //Load the first item(s) to display
      //Set a query according to time in milliseconds
      mQuery = mDatabase.collection("Announcements")
              .orderBy("time", Query.Direction.DESCENDING)
              .limit(5);

      //Getting all documents under Announcement collection with query's condition

  annon_listener = mQuery.addSnapshotListener(new EventListener<QuerySnapshot>() {
      @Override
      public void onEvent(final QuerySnapshot documentSnapshots, FirebaseFirestoreException e) {

          //If something went wrong
          if (e != null)
              Log.w(TAG, "Listen failed.", e);


          //If any post exist put it to model and add it to List to populate the CardView
          //If data exist in the first 5 items then item should be loaded making our 'isFirstListLoaded' variable to be true
          if (documentSnapshots != null && !documentSnapshots.isEmpty()){

              //If first item are loaded then every update post should be on the top not at the bottom
              //This can only be called once to avoid confusion/duplication getting new item
              if (isFirstListLoaded){
                  //Get the documents of last item listed in our RecyclerView
                  mLastSeen = documentSnapshots.getDocuments().get(documentSnapshots.size()-1);
                  //Clear the list first to get a latest data
                  announcementList.clear();
              }

              //Loop to read each document
              for (DocumentChange doc : documentSnapshots.getDocumentChanges()){

                  //Only added document will be read
                  switch (doc.getType()){

                      case ADDED:

                          //This can only be called once to avoid confusion getting new item(s)
                          if (isFirstListLoaded){
                              //Call the model to populate it with document
                              AnnouncementModel annonPost = doc.getDocument().toObject(AnnouncementModel.class)
                                      .withId(doc.getDocument().getId());

                              announcementList.add(annonPost);
                              announcementRecyclerAdapter.notifyDataSetChanged();
                              noContent.setVisibility(View.GONE);
                              label.setVisibility(View.VISIBLE);
                          }

                          //This will be called once a user added new item to database and put it to top of the list
                          else if (!isFirstListLoaded){

                              if (containsLocation(announcementList, doc.getDocument().getId() )){
                                  Log.d(TAG, "Items are gonna duplicate!");

                              }
                              else{
                                  //Call the model to populate it with document
                                  AnnouncementModel annonPost = doc.getDocument().toObject(AnnouncementModel.class)
                                          .withId(doc.getDocument().getId());

                                  //This will be called only if user added some new post
                                  announcementList.add(0, annonPost);
                                  announcementRecyclerAdapter.notifyItemInserted(0);
                                  announcementRecyclerAdapter.notifyItemRangeChanged(0, announcementList.size());
                              }
                          }

                          //Just checking of where's the data fetched from
                          String source = documentSnapshots.getMetadata().isFromCache() ?
                                  "Local" : "Server";

                          Log.d(TAG, "Data fetched from " + source + "\n" + doc.getDocument().getData());

                          break;
                  }

              }
              //After the first item/latest post was loaded set it to false it means that first items are already fetched
              isFirstListLoaded = false;
          }

      }
  });


delete_update_listener = mDatabase.collection("Announcements").addSnapshotListener(new EventListener<QuerySnapshot>() {
      @Override
      public void onEvent(@javax.annotation.Nullable QuerySnapshot queryDocumentSnapshots, @javax.annotation.Nullable FirebaseFirestoreException e) {

          //If something went wrong
          if (e != null)
              Log.w(TAG, "Listen failed.", e);

          if (queryDocumentSnapshots != null && !queryDocumentSnapshots.isEmpty()) {
              //Instead of simply using the entire query snapshot
              //See the actual changes to query results between query snapshots (added, removed, and modified)
              for (DocumentChange doc : queryDocumentSnapshots.getDocumentChanges()) {
                  switch (doc.getType()) {
                      case MODIFIED:
                          Log.d(TAG, "Modified city: " + doc.getDocument().getData());
                          break;
                      case REMOVED:
                          //Get the document ID of post in FireStore
                          //Perform a loop and scan the list of announcement to target the correct index
                          for(int i = 0; i < announcementList.size(); i++) {
                              //Check if the deleted document ID is equal or exist in the list of announcement
                              if(doc.getDocument().getId().equals(announcementList.get(i).AnnouncementsID)) {
                                  int prevSize = announcementList.size();
                                  //If yes then delete that object in list by targeting its index
                                  Log.d(TAG, "Removed city: " + announcementList.get(i).getTitle());
                                  announcementList.remove(i);
                                  //Notify the adapter that some item gets remove
                                  announcementRecyclerAdapter.notifyItemRemoved(i);
                                  announcementRecyclerAdapter.notifyItemRangeChanged(i,prevSize-i);
                                  break;
                              }
                          }
                          break;
                  }
              }
          }

      }
  });



//Load more queries
    private void loadMoreList() {
        //Load the next item(s) to display
        //Set a query according to time in milliseconds
        //This time start getting data AFTER the last item(s) loaded
        if (mLastSeen != null)
            mQuery = mDatabase.collection("Announcements")
                    .orderBy("time", Query.Direction.DESCENDING)
                    .startAfter(mLastSeen)
                    .limit(5);

        //Getting all documents under Announcement collection with query's condition
        annon_listener = mQuery.addSnapshotListener(new EventListener<QuerySnapshot>() {
            @Override
            public void onEvent(final QuerySnapshot documentSnapshots, FirebaseFirestoreException e) {
                //If something went wrong
                if (e != null)
                    Log.w(TAG, "Listen failed.", e);

                if (documentSnapshots != null && !documentSnapshots.isEmpty()) {
                    //If more data exist then update our 'mLastSeen' data
                    //Update the last list shown in our RecyclerView
                    mLastSeen = documentSnapshots.getDocuments().get(documentSnapshots.size() - 1);

                    //Loop to read each document
                    for (DocumentChange doc : documentSnapshots.getDocumentChanges()) {
                        //Only added document will be read
                        switch (doc.getType()) {

                            case ADDED:
                                //Call the model to repopulate it with document
                                AnnouncementModel annonPost = doc.getDocument().toObject(AnnouncementModel.class)
                                        .withId(doc.getDocument().getId());

                                int prevSize = announcementList.size();

                                //Add any new item(s) to the List
                                announcementList.add(annonPost);
                                //Update the Recycler adapter that new data is added
                                //This trick performs recycling even though we set nested scroll to false
                                announcementRecyclerAdapter.notifyItemInserted(prevSize);
                                announcementRecyclerAdapter.notifyItemRangeInserted(prevSize, 5);

                                //Just checking of where's the data fetched from
                                String source = documentSnapshots.getMetadata().isFromCache() ?
                                        "Local" : "Server";

                                Log.d(TAG, "Data LOADED from " + source + "\n" + doc.getDocument().getData());
                                break;

                            case REMOVED:
                                break;

                            case MODIFIED:

                                break;

                        }

                    }
                }

                //If no more item(s) to load
                else if (!isDetached() && getContext() != null)
                    StyleableToast.makeText(getContext(), "All items are loaded.", R.style.mytoastNoItems).show();

            }
        });
    }

Additionally I tried to observe how doc types work "ADDED" , "REMOVED", and "MODIFIED". If I put "REMOVED" also inside the listeners that is using the query,the REMOVED is the one that is being called first followed by ADDED when adding new items which will cause more trouble.

  • 2
    **[This](https://stackoverflow.com/questions/50741958/how-to-paginate-firestore-with-android)** is a recommended way in which you can paginate queries by combining query cursors with the limit() method. I also recommend you take a look at this **[video](https://www.youtube.com/watch?v=KdgKvLll07s)** for a better understanding. – Alex Mamo Jun 12 '18 at 06:39
  • Could you elaborate what's the main difference of the example to what I have? I think there's no differences at all on how it load next batches. The problem I'm facing right now is performing delete on the collection which is causing the item to get rumbled. –  Jun 12 '18 at 06:47
  • You are using a SnapshotListener, which tries to get the data in realtime while in that example is used only a get() call. So you can notify the adapter only when you have deleted an item. – Alex Mamo Jun 12 '18 at 07:08
  • I still want to make it in real time so should I use just one Listener that will listen for added, removed, and delete while the paginating use get()? –  Jun 12 '18 at 07:45
  • Give it a try, in this way. – Alex Mamo Jun 12 '18 at 07:46
  • Sure thanks for the help. –  Jun 12 '18 at 07:50
  • Wow I think your approach is much cleaner and simple, I already think to not use SnapshotListener before but I just do not know how to get data with queries in a single request like get() and onComplete. Never thought that task.getResult has that capability too. Anyway your approach is working except when I add "ADDED" on my listener for new inserted post purposes it duplicates the data when scrolling like this first launch: 1,2,3,4,5,6,7,8,9 after srcoll to load more item result : 1,2,3,4,5,6,7,8,9,1,2,3,4,5... –  Jun 12 '18 at 09:21
  • Deletion is now working but Adding and Scrolling is not working now. –  Jun 12 '18 at 09:23
  • You can simply use get() as in the [official doc](https://firebase.google.com/docs/firestore/query-data/get-data#get_multiple_documents_from_a_collection) or in [here](https://firebase.google.com/docs/firestore/query-data/query-cursors#paginate_a_query). If you have that behaviour it means that you are not actually getting the last visible item, you are getting the same one (the first). Adding should work in the same way. Scrolling in the same way. Please see the video to see how scrollin works. (It's in final part) – Alex Mamo Jun 12 '18 at 09:26
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/172971/discussion-between-archeremiya-and-alex-mamo). –  Jun 12 '18 at 12:39

2 Answers2

3

So finally after a months I figure it out the best way and best approach to perform Real-time updates with FireStore with RecyclerView. With the help of Alex Mamo's answer here

The best way is to get the data/document once, then provide a ListenerRegistration with that collection. Here is my solution.

First is to initialize a member boolean variable and set it to true. This is necessary because doc type ADDED is being triggered in the first launch and we do not need that.

private boolean isFirstListLoaded = true;

Next is to declare your ListenerRegistration, this is an optional but I highly recommend to provide a listener so that you won't be needed anymore to inlcude 'this' in addSnapshotListener parameter. Including 'this' to the parameter will save you some memory at data but also sometimes will stop the Real-time function since it depends on Fragment or Activity Lifecycle which destroy the purpose of having a Real-time updates.

 private ListenerRegistration update_listener

Then create your queries like this

 private Query mQuery;

Ascending or Descending and limit is depends on you.

Put this on your onCreate method so it will run 1 time only.

mQuery= mDatabase.collection("Your Collection")
                .orderBy("some fields within each document like names or time", Query.Direction.DESCENDING)
                .limit(5);


  //Run the first query in the beginning
        mQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
            @Override
            public void onComplete(@NonNull Task<QuerySnapshot> task) {
                if (task.isSuccessful()) {
                    if (!task.getResult().isEmpty()) {

                        //Get the documents of last item listed in our RecyclerView
                        mLastSeen = task.getResult().getDocuments().get(task.getResult().size() - 1);

                        //Loop to read each document
                        for (DocumentSnapshot document : task.getResult()) {

                            //Call the model to populate it with document
                            Model model = Objects.requireNonNull(document.toObject(Model .class))
                                    .withId(document.getId());
                            //Add every item/document to the list
                            mList.add(model);
                            //Notify the adapter that new item is added
                            yourRecyclerAdapter.notifyItemInserted(mList.size());
                            noContent.setVisibility(View.GONE);
                            label.setVisibility(View.VISIBLE);

                            //Just checking of where's the data fetched from
                            String source = document.getMetadata().isFromCache() ?
                                    "Local" : "Server";

                            Log.d(TAG, "Data fetched from " + source + "\n" + document.getData());
                        }
                     }

                    //If task is successful even though there's no existing item yet, then first fetch is success
                isFirstListLoaded = false;
                }
                else if (getContext() != null)
                    Toast.makeText(getContext(),"Error: "+ Objects.requireNonNull(task.getException()).getMessage(),Toast.LENGTH_LONG).show();

            }
        });

Also this.

 //Listener
    update_listener = mDatabase.collection("Announcements").addSnapshotListener(new EventListener<QuerySnapshot>() {
        @Override
        public void onEvent(@javax.annotation.Nullable QuerySnapshot queryDocumentSnapshots, @javax.annotation.Nullable FirebaseFirestoreException e) {

            //If something went wrong
            if (e != null)
                Log.w(TAG, "Listen failed.", e);

            if (queryDocumentSnapshots != null) {
                //Instead of simply using the entire query snapshot
                //See the actual changes to query results between query snapshots (added, removed, and modified)
                for (DocumentChange doc : queryDocumentSnapshots.getDocumentChanges()) {
                    switch (doc.getType()) {

                        case ADDED:

                            if (!isFirstListLoaded){
                                //Call the model to populate it with document
                                Model model= doc.getDocument().toObject(Model.class)
                                        .withId(doc.getDocument().getId());

                                //This will be called only if user added some new post
                                mList.add(0, model);
                                yourRecyclerAdapter.notifyItemInserted(0);
                                yourRecyclerAdapter.notifyItemRangeChanged(0, announcementList.size());
                            }

                            break;

                        case MODIFIED:
                            break;

                        case REMOVED:
                            //Get the document ID of post in FireStore
                            //Perform a loop and scan the list of announcement to target the correct index
                            for (int i = 0; i < announcementList.size(); i++) {
                                //Check if the deleted document ID is equal or exist in the list of announcement
                                if (doc.getDocument().getId().equals(announcementList.get(i).AnnouncementsID)) {
                                    //If yes then delete that object in list by targeting its index
                                    Log.d(TAG, "Removed Post: " + announcementList.get(i).getTitle());
                                    announcementList.remove(i);
                                    //Notify the adapter that some item gets remove
                                    announcementRecyclerAdapter.notifyItemRemoved(i);
                                    announcementRecyclerAdapter.notifyItemRangeChanged(i, announcementList.size());
                                    break;
                                }
                            }
                            break;
                    }
                }

                isFirstListLoaded = false;
            }

        }
    });

Then whenever you want to load more items call this method.

  private void loadMoreList() {
    //Load the next item(s) to display
    //Set a query according to time in milliseconds
    //This time start getting data AFTER the last item(s) loaded
    if (mAnnonLastSeen != null)
        mAnnouncementQuery = mDatabase.collection("Your collection")
                .orderBy("some field within your documents", Query.Direction.DESCENDING)
                .startAfter(mLastSeen)
                .limit(5);

    mAnnouncementQuery.get().addOnCompleteListener(new OnCompleteListener<QuerySnapshot>() {
        @Override
        public void onComplete(@NonNull Task<QuerySnapshot> task) {
            if (task.isSuccessful()) {
                if (!task.getResult().isEmpty()) {
                    //Get the documents of last item listed in our RecyclerView
                    mLastSeen = task.getResult().getDocuments().get(task.getResult().size() - 1);

                    //Loop to read each document
                    for (DocumentSnapshot document : task.getResult()) {

                        //Call the model to populate it with document
                        AnnouncementModel annonPost = Objects.requireNonNull(document.toObject(AnnouncementModel.class))
                                .withId(document.getId());

                        //Add any new item(s) to the List
                        announcementList.add(annonPost);
                        //Update the Recycler adapter that new data is added
                        //This trick performs recycling even though we set nested scroll to false
                        announcementRecyclerAdapter.notifyItemInserted(announcementList.size());

                        //Just checking of where's the data fetched from
                        String source = document.getMetadata().isFromCache() ?
                                "Local" : "Server";

                        Log.d(TAG, "Data fetched from " + source + "\n" + document.getData());
                    }

                } else if (!isDetached() && getContext() != null)
                    Toast.makeText(getContext(), "All items are loaded.", Toast.LENGTH_LONG).show();

            }
            else if (getContext() != null)
                Toast.makeText(getContext(),"Error: "+ Objects.requireNonNull(task.getException()).getMessage(), Toast.LENGTH_LONG).show();

        }
    });
}

Where mLastSeen is a member varialble DocumentSnapshot. Cheers!

0

It seems like you're building the list out of segments, and each segment:

  • starts at a specific document
  • contains the next 5 items

In that case removing an item from a segment, results in an additional change to the start document of the next segment.

Although it's not too bad on the loads from he server, since most documents will come from the local cache, this does lead to quite some shuffling of documents.

For this reason you'll find many developers taking alternative approaches. The most common one I see is to only have a single segment, and simply increase the limit as the user scroll down. So the query initially has 5 items, then 10, then 15, etc.

A more complex scenario would be to anchor each segment at a starting document, and end document. That way removing a document from within a segment, won't change the other segments around it. But this scenario has other complications, so I'd definitely go for something more well known first.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • Yeah I suffer from this in weeks, when the first 5 item is loaded then I delete in one of those item, the query itself add a new item which is the 6th item and since all newest item will be added at the top "0" then item will be 6, 1,2,4,5 then 6 again. I guess its because of local cache? –  Jun 12 '18 at 05:06
  • Could anyone give a better approach for this? Its the only way I know as of now –  Jun 12 '18 at 05:20