android - querying keys of each node from list with Firebase

Question

I have a list of string and this list contains node names in Firebase database.

And I have a custom sort method therefore I want to get the all of the keys for each node in my custom list without using Firebase` query sorting functions. And because of it I need all data to be retrieved, so adding to list as it retrieves some data is not an option.

However the problem starts with asynchronous structure of Firebase.

I iterate through every node name in my custom list and under that loop I create another loop in Firebase` thread ( addListenerForSingleValueEvent ) to retrieve all keys. But it works asynchronously. I've tried to change it to synchronous by using Thread and Semaphore but it didn't work. I've also used custom interface (FirebaseDataRetrieveListener) to indicate when the loop in valueEventListener finishes but since valueEventListener instantly return this is not possible without pausing the thread.

If Task's can be used in this situation, how it could be, or are there any other solutions?

private void getKeysFromNodeList(final FirebaseDataRetrieveListener listener) //TODO [BUG] when first for loop initializes it ignores the seperate thread in the inner anonymous class so it'll be already finished it's cycle before Firebase` loop starts...
    {
        listener.onStart();

                final DatabaseReference databaseRef = firebaseInstance.rootRef.child(context.getString(R.string.databaseref));
                for (iterator = 0; iterator < nodeList.size(); iterator ++)
                    {

                        Query query = databaseRed.child(nodeList.get(iterator));

                        query.addListenerForSingleValueEvent(new ValueEventListener()
                            {
                                @Override
                                public void onDataChange(@NonNull DataSnapshot dataSnapshot)
                                    {


                                        for (DataSnapshot snap : dataSnapshot.getChildren())
                                            {

                                                iterator++;

                                                String key= snap.getKey();
                                                // I've done my jobs with given key here


                                                // if got the last key from node and iterated the last node
                                                if (firebaseIterator== dataSnapshot.getChildrenCount() - 1 && iterator == nodeList.size() - 1)
                                                    {


                                                        firebaseIterator= 0;// reset the iterators
                                                        iterator= 0;

                                                        listener.onSuccess();
                                                        break;
                                                    }
                                                // if got the last key from node
                                                else if (firebaseIterator== dataSnapshot.getChildrenCount() - 1)
                                                    {
                                                        firebaseIterator= 0; // reset the iterator

                                                        break;
                                                    }
                                            }
                                    }

                                @Override
                                public void onCancelled(@NonNull DatabaseError databaseError)
                                    {
                                        listener.onFailed(databaseError);
                                        return;
                                    }
                            });


                    }
                ;


            } 

    }

FirebaseDataRetrieveListener.java

public interface FirebaseDataRetrieveListener
{
    public void onStart();
    public void onSuccess();
    public void onFailed(DatabaseError databaseError);
}

Answer 1

Basically, you're trying to return a value synchronously from an API that's asynchronous. That's not a good idea. You should handle the APIs asynchronously as intended. Pausing a thread might solve your problem but only partial. and it's not recommened at all.

Firebase APIs are asynchronous, meaning that onDataChange() method returns immediately after it's invoked, and the callback from the Task it returns, will be called some time later. There are no guarantees about how long it will take. So it may take from a few hundred milliseconds to a few seconds before that data is available. Because that method returns immediately, the values that are coming from the database will not have been populated from the callback yet.

A quick solve for this problem would be to use those values only inside the callback, otherwise I recommend you see the last part of my anwser from this post in which I have explained how it can be done using a custom callback. You can also take a look at this video for a better understanding.

As a conclusion, there is no way to turn an asynchronously API as Firebase is into a synchronous one.

Answer 2

To solve this problem I made my method recursive and used a listener. In my solution we are gonna keep track on our how many times we called the recursive method, and how many time Firebase looped through to keys of given node in order to detect whether we reach to end of node and we looped all the nodes in the list.

Declare 2 iterator and 1 listener in your class` scope eg.

        private RecursiveListener listener;
        private long firebaseIterator = 0;
        private int recursiveIterator = 0;

        public interface getKeysFromNodeListListener
            {
                void onFinished();
            }

We are gonna use the recursive listener in order to get notified whether our recursive function has finished.

Before calling your method define the listener and override onFinished(). onFinished gonna be called when we both looped through all nodes and their keys.

private  void getKeysFromNodeList(final getKeysFromNodeListListener listener)
        {

final DatabaseReference databaseRef = firebaseInstance.rootRef.child(database_reference);

                    if (recursiveIterator < nodesList.size())
                        {

                            Query query = databaseRef.child(nodesList.get(recursiveIterator));
                            query.addListenerForSingleValueEvent(new ValueEventListener()
                                {
                                    @Override
                                    public void onDataChange(@NonNull DataSnapshot dataSnapshot)
                                        {
                                            for (DataSnapshot snap : dataSnapshot.getChildren())
                                                {
                                                    firebaseIterator++;


                                                    //Do you job with the current key
                                                    String key = snap.getKey();
                                                    keysList.add(key);

                                                    // if got the last key from key and iterated the last node
                                                    if (firebaseIterator == dataSnapshot.getChildrenCount()  && recursiveIterator == nodeList.size() -1)
                                                        {
                                                            firebaseIterator = 0;// reset the iterators
                                                            recursiveIterator = 0;
                                                            return;
                                                        }
                                                    // if got the last key from current node
                                                    else if (firebaseIterator == dataSnapshot.getChildrenCount() )
                                                        {
                                                            recursiveIterator++;  // increase the recursive iterator because we looped through all the keys under the current node
                                                            firebaseIterator = 0; // reset the recursiveIterator because we gonna need to loop again with default value
                                                            getKeysFromNodeList(listener);  // call the same method
                                                        }
                                                }
                                        }

                                    @Override
                                    public void onCancelled(@NonNull DatabaseError databaseError)
                                        {
                                            Log.e(TAG, "Firebase query failed " + databaseError);
                                            return;
                                        }
                                });
                        }
                }

And when using the method,first create a listener and override the onFinished() and then call the method.

   listener = new getKeysFromNodeListListener()
                                    {
                                        @Override
                                        public void onFinished()
                                            {

                                             // Do whatever you gonna do after you got all the keys from all nodes.
                                            }
                                    };
                                 getKeysFromNodeList(listener);