Sorting data while using distributed counter

Question

In the Firestore guide it's mentioned that one should use distributed counters to handle multiple updates of a single document at the same time.

This got me thinking. With this counter implementation how can you sort your data? For example let's say that I have a database of posts and each post has its own distributed counter for upvotes. Now, I'd like to get maybe 20 most upvoted posts. How can you do that?

From my understanding of Firestore database this can't be done. Unless I'm missing something? Is there a better solution for counters for such use case I described above?

score 6 · Accepted Answer · answered Jan 26 '18 at 14:47

The main problem with counters is this limit:

In Cloud Firestore, you can only update a single document about once per second, which might be too low for some high-traffic applications.

The same documentation page that explains how to use distributed counters to work around this, also shows an example of how to then read the counter totals:

To get the total count, query for all shards and sum their count fields:

function getCount(ref) {
    // Sum the count of each shard in the subcollection
    return ref.collection('shards').get().then(snapshot => {
        let total_count = 0;
        snapshot.forEach(doc => {
            total_count += doc.data().count;
        });

        return total_count;
    });
}

It also mentions the main disadvantage of this:

Cost - The cost of reading a counter value increases linearly with the number of shards, because the entire shards subcollection must be loaded.

One way to work around this, is to periodically read the counts from the shards and update a master list of counts. This essentially turns the whole exercise into a map-reduce solution. You'll want to run this reduce-code on a schedule, since otherwise, you'd still run into the write rate limit. Using a periodically triggered Cloud Function sounds ideal for this.

score 0 · Answer 2 · answered Mar 20 '19 at 02:37

According to this post, an alternative option is to use realtime database to replace the distributed counter on firestore, and still need to periodically read the counts from realtime database and write it back to firestore.

Not sure if this is a more scalable way than distributed counter.

Sorting data while using distributed counter

2 Answers2