Firebase count group by

Question

Does Firebase supports grouped counting?
I would like to get counting for specific key grouped by the value.

Example of my data structure:

"playbackPosition" : {
    "-JZ2c7v-imQXaEx2c4Cs" : {
        "data" : "0"
    },
    "-JZ2cDta73UGAgwIOejL" : {
        "data" : "25"
    },
    "-JZ2cJutDZu-7quLpLI2" : {
        "data" : "50"
    },
    "-JZ2cO-Me0nGK5XLTwd-" : {
        "data" : "75"
    },
    "-JZ2cSh-XnstYxbE_Zad" : {
        "data" : "100"
    },
    "-JZ2cWxya0-kmTPCze4u" : {
        "data" : "0"
    },
    "-JZ2c_wX-ODv43TKXkNG" : {
        "data" : "25"
    }
}

Required results based on the data key :

0 => 2  
25 => 2  
50 => 1  
75 => 1  
100 => 1

And of course I must consider that it will have thousands of children's, not only 7...

Thanks ahead!

EDIT
Deeper explanation of the app and the problem we want to solve.

We have video scripts which runs on different websites, each video session (a user session) sends events and data, you can see an example here, check the Network tab - https://secure.bwebi.co/videostir/regular.html

Our goal is to collect this data and create an analytics real time dashboard with few charts and graphs.

You can see our current data structure here

Example for graphs we need:
Completion rate
General - Bar graph showing overall number of views per clip duration pre defined periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations/ ignore silent.
X axis - groups 0-25, 25-50,50-75,75-99, 100
Y axis - number of views

Views per day (with completion rate)
General - Multi lines graph showing number of views per day per duration periods.
Filters - date (start/end), unique/all, All urls / specific urls (embed on), All configurations / specific configurations / ignore silent.
X axis - Time in days
Y axis - Number of views
Lines for:
Total daily views
Daily views with 100% duration
Daily views with 75-99% duration
Daily views with 50-75% duration
Daily views with 25-50% duration
Daily views with 0-25% duration

Hope it's more clear now!

No, it doesn't. You'll have to roll your own. For some example (though not a group-by) see these blog posts: https://www.firebase.com/blog/2013-10-01-queries-part-one.html and https://www.firebase.com/blog/2014-01-02-queries-part-two.html — Frank van Puffelen, Oct 13 '14 at 11:38
nope. you can use index. in your example you will have only 5 lists. — webduvet, Oct 13 '14 at 14:32
I would choose a tool like Keen.io over Firebase for storing metrics and analytics. These tools are built exactly for this purpose and also have simple APIs for quickly writing stats from the client. — Kato, Oct 14 '14 at 14:38
To clarify the difference, these analytics are really just a write-heavy and then computational process. You don't synchronize them to thousands of users simultaneously. I.e. status engines are write-heavy, provide built-in algorithms for generating metrics, and store tons of data; Firebase provides real-time, lightning fast synchronization of data and free asset hosting, but none of those stats-related capabilities out of the box. — Kato, Oct 15 '14 at 14:35
Thank you @Kato, I really appreciate you deep and knowledgeable answers! They helped me a lot and now I have a deeper understanding of what kind of tool we need. Cheers! :) — Chen Alon, Oct 15 '14 at 16:15

score 14 · Accepted Answer · edited Mar 20 '17 at 10:29

Group by is a SQL function. The reason SQL can't do real-time data is because this sort of method does not scale. Mongo provides similar functionality, but once again, it doesn't scale. You may notice a pattern here of why Firebase does not provide this sort of query function.

It would be extremely helpful if you provided some context of what you're actually attempting to accomplish here, what the rules of the app are, and what approaches you've ruled out, rather than just your presupposed solution of group by. There are probably other, possibly better, alternatives. See the XY problem.

Here are a couple generic alternatives derived by making sweeping assumptions about your use case.

Store the totals

This is the most scalable solution. Store your data as follows:

/playbacks/$id/<playback data>
/group_totals/$group/<count>

When writing to playbacks, also update the count for the appropriate group:

var fb = new Firebase(URL);
function addPlayback(rec) {
   var ref = fb.child('playbacks').push(rec, function(err) {
      if( err )  throw err;
      incrementCount(rec.data);
   });
}

function incrementCount(count) {
   fb.child('group_totals/' + count).transaction(function(currentVal) {
     return (currentVal||0)+1;
   });
}

Now when you want to get the total for a group, you can simply look up the value at group_totals/$group. Similarly, you can store ids for records that belong to each group and utilize that index to grab only the records for a given group.

Use priorities to fetch and group

A simpler approach would be to give each record a priority based on the group/data value.

var fb = new Firebase(URL);
function addPlayback(rec) {
   rec['.priority'] = rec.data;
   var ref = fb.child('playbacks').push(rec, function(err) {
      if( err )  throw err;
   });
}

Now to grab a set of records for a given group:

var fb = new Firebase(URL);
function getGroup(groupValue, callback) {
   fb.child('playbackPosition').startAt(groupValue).endAt(groupValue).once('value', callback);
}

function logGroupCount(groupValue, callback) {
   getGroup(groupValue, function(snap) {
       console.log('there are ' + snap.numChildren() + ' items in group ' +groupValue);
   });
}

Awesome answer! Thanks a lot! I edited my question, hope it will be more clear now. — Chen Alon, Oct 14 '14 at 06:57

score 0 · Answer 2 · answered May 24 '21 at 19:08

A am not professional programmer, I am just learning. Here is the piece of code I came up with when I wanted to group my query results:

 btn.setOnClickListener(new View.OnClickListener() {
            @Override
            public void onClick(View v) {
                progressBar.setVisibility(View.VISIBLE);
                tv_info.setText("Please wait ... ");

                Query query = collectionRef.orderBy("timestamp", Query.Direction.DESCENDING);

                query.get().addOnSuccessListener(new OnSuccessListener<QuerySnapshot>() {
                    @Override
                    public void onSuccess(QuerySnapshot queryDocumentSnapshots) {
                        String dataInfo = "";
                        int arrayLength = 1;
                        List <String> UsedNames = new ArrayList<String>();
                        String someName = "...";

                        String oneRow = "";
                        int orderNumber = -1;
                        int count = 0;

                        for (QueryDocumentSnapshot documentSnapshot : queryDocumentSnapshots) {
                            OneRecord oneRecord = documentSnapshot.toObject(OneRecord.class);
                            someName = oneRecord.getSomeName();

                            if (UsedNames.contains(someName)) {
//                                Log.i("test" , "Array Contains ");
                            } else {
                                orderNumber += 1;
                                UsedNames.add(someName);

                            }

                   }

                        List list = queryDocumentSnapshots.toObjects(OneRecord.class);


                        for (String someString : UsedNames) {

                            int counter = 0;
                            for (int i = 0; i < list.size(); i++) {
                                OneRecord oneRecord = (OneRecord) list.get(i);
                                String name = oneRecord.getName();

                                if (someString.equals(name)) {
                                    counter += 1;
                                }
                            }
                            Log.i("test" , "Array: " + someString + " : " + counter);
                            count = count +1;
                            dataInfo = dataInfo + someString + " : " + counter + "\n";

                        }



                        Log.i("test" , "Used length: " + UsedNames.size());


                        progressBar.setVisibility(View.GONE);
                        tv_info.setText(dataInfo);


                    }
                }).addOnFailureListener(new OnFailureListener() {
                    @Override
                    public void onFailure(@NonNull Exception e) {
                        progressBar.setVisibility(View.GONE);
                        tv_info.setText("Could not query last records: " + e.getMessage());
                    }
                });


            }
        });

Unfortunately I did not figure out how to sort them in DESCENDING or ASCENDING order

Firebase count group by

2 Answers2

Store the totals

Use priorities to fetch and group

Linked