323

Is it possible to count how many items a collection has using the new Firebase database, Cloud Firestore?

If so, how do I do that?

Dan McGrath
  • 41,220
  • 11
  • 99
  • 130
Guilherme Torres Castro
  • 15,135
  • 7
  • 59
  • 96
  • 11
    Possible duplicate of [How to get a count of number of documents in a collection with Cloud Firestore](https://stackoverflow.com/questions/46553314/how-to-get-a-count-of-number-of-documents-in-a-collection-with-cloud-firestore) – Dan McGrath Oct 03 '17 at 22:07
  • I think you might also be interested in this article, [How to count the number of documents in a Firestore collection?](https://medium.com/firebase-tips-tricks/how-to-count-the-number-of-documents-in-a-firestore-collection-3bd0c719978f). – Alex Mamo Jul 21 '21 at 08:12
  • One of the newer answers should become the accepted answer for this, since there's now a direct COUNT query option https://www.reddit.com/r/googlecloud/comments/y1qrwm/firestore_write_throughput_and_concurrency_limits/ – grunet Oct 14 '22 at 01:54
  • count() is now natively supported in Firestore: https://stackoverflow.com/a/74046678/5861618 – Rosário Pereira Fernandes Oct 14 '22 at 16:04
  • It is now possible with the new Firebase Aggregated Queries (Simply write .count().get(); after your query.) – mik Oct 24 '22 at 09:38

30 Answers30

362

2023 Update

Firestore now supports aggregation queries.

Node SDK

const collectionRef = db.collection('cities');
const snapshot = await collectionRef.count().get();
console.log(snapshot.data().count);

Web v9 SDK

const coll = collection(db, "cities");
const snapshot = await getCountFromServer(coll);
console.log('count: ', snapshot.data().count);

Notable Limitation - You cannot currently use count() queries with real-time listeners and offline queries. (See below for alternatives)

Pricing - Pricing depends on the number of matched index entries rather than the number of documents. One index entry contains multiple documents, making this cheaper than counting documents individually.


Old Answer

As with many questions, the answer is - It depends.

You should be very careful when handling large amounts of data on the front end. On top of making your front end feel sluggish, Firestore also charges you $0.60 per million reads you make.


Small collection (less than 100 documents)

Use with care - Frontend user experience may take a hit

Handling this on the front end should be fine as long as you are not doing too much logic with this returned array.

db.collection('...').get().then(snap => {
  size = snap.size // will return the collection size
});

Medium collection (100 to 1000 documents)

Use with care - Firestore read invocations may cost a lot

Handling this on the front end is not feasible, as it has too much potential to slow down the user's system. We should handle this logic server side and only return the size.

The drawback to this method is you are still invoking Firestore reads (equal to the size of your collection), which in the long run may end up costing you more than expected.

Cloud Function:

db.collection('...').get().then(snap => {
  res.status(200).send({length: snap.size});
});

Front End:

yourHttpClient.post(yourCloudFunctionUrl).toPromise().then(snap => {
   size = snap.length // will return the collection size
})

Large collection (1000+ documents)

Most scalable solution


FieldValue.increment()

As of April 2019 Firestore now allows incrementing counters, completely atomically, and without reading the data prior. This ensures we have correct counter values even when updating from multiple sources simultaneously (previously solved using transactions), while also reducing the number of database reads we perform.


By listening to any document deletes or creates, we can add to or remove from a count field that is sitting in the database.

See the Firestore docs - Distributed Counters Or have a look at Data Aggregation by Jeff Delaney. His guides are truly fantastic for anyone using AngularFire, but his lessons should carry over to other frameworks as well.

Cloud Function:

export const documentWriteListener = functions.firestore
  .document('collection/{documentUid}')
  .onWrite((change, context) => {

    if (!change.before.exists) {
      // New document Created : add one to count
      db.doc(docRef).update({ numberOfDocs: FieldValue.increment(1) });
    } else if (change.before.exists && change.after.exists) {
      // Updating existing document : Do nothing
    } else if (!change.after.exists) {
      // Deleting document : subtract one from count
      db.doc(docRef).update({ numberOfDocs: FieldValue.increment(-1) });
    }

    return;
  });

Now on the frontend you can just query this numberOfDocs field to get the size of the collection.

Mises
  • 4,251
  • 2
  • 19
  • 32
Matthew Mullin
  • 7,116
  • 4
  • 21
  • 35
  • 3
    These methods are using a recount of the number of records. If you use a counter and increment the counter using a transaction, would that not achieve the same result without the added cost and need of a cloud function ? – user3836415 Jun 17 '18 at 23:39
  • 33
    The solution for large collections is not idempotent and does not work at any scale. Firestore document triggers are guaranteed to run at least once, but may run multiple times. When this happens, even maintaining the update inside a transaction can run more than one time, which will give you a false number. When I tried this, I ran into issues with fewer than a dozen document creations at a time. – Tym Pollack Dec 29 '18 at 19:30
  • 3
    Hi @TymPollack. I have noticed some inconsistent behaviour using cloud triggers. Any chance you could link me to and article or forum to explain the behaviour you have experienced? – Matthew Mullin Jan 18 '19 at 11:24
  • Hi @MatthewMullin, right now when I do `event.data.previous` gives me an error saying "Property 'data' does not exist on type 'Change'", any idea?, also the same error when I do `event.data.exists`. –  Jan 29 '19 at 14:55
  • @RobertGomez Hi Robert, Firebase Functions have since upgraded to v1 where a few changes were made. Have a look at their migration guide here https://firebase.google.com/docs/functions/beta-v1-diff. Ill update my answer when I have a moment later – Matthew Mullin Jan 29 '19 at 14:58
  • @MatthewMullin Thanks for answering me back. I will read it right now, also I forgot to mention that `snap.usersCount` also gives me error, it says "Property 'usersCount' does not exist on type 'DocumentSnapshot'". Now we need to access the property like `snap.data.usersCount`? I'm confused. –  Jan 29 '19 at 15:06
  • @RobertGomez Yes, its snap.data().. My answer should be up to date in the next couple minites – Matthew Mullin Jan 29 '19 at 15:13
  • @MatthewMullin I did `.update({usersCount: snap.data().usersCount + 1});` but it gives me the error "Object is possibly 'undefined'". I think I will wait for your edition to see what I'm doing wrong. –  Jan 29 '19 at 15:19
  • Just an update to this: I don't believe the information about Firebase/Firestore pricing is correct here. As stated in the Google Docs, returning a count of a list of documents in a collection counts as one read, not 1 per document. See here: [Google Pricing](https://firebase.google.com/docs/firestore/pricing) specifically the section "Queries other than document reads". So just get a list of collection ids and then count that. – cmprogram Sep 01 '19 at 13:56
  • 3
    @cmprogram you're reading the entire collection and data when when using db.collection('...')... so when you don't need the data then you're right - you could easily request for a list of collection IDs (not collection documents data) and it counts as one read. – atereshkov Sep 04 '19 at 09:50
  • 2
    @MatthewMullin can you provide a frontend code example to access the numberOfDocs field? I don't understand if that field is in the collection reference or in another collection like 'counters'. thanks!! – Damsorian Dec 02 '19 at 23:07
  • Beware triggers might take some time to execute - I've experienced it myself recently - so I'd account for some inconsistency between the last write and the actual updated count. I think this is what @MatthewMullin is also experiencing. – santamanno Mar 28 '20 at 23:22
  • There are actually both an `.onCreate()` and an `.onDelete()` function. Why not use these instead? :) Otherwise, good job! – Karolina Hagegård Apr 06 '21 at 09:49
  • 1
    @KarolinaHagegård You definitely could! This method just puts them all in one place for simplicity. Separating them out will also reduce unnecessary Update triggers could add to your bill. – Matthew Mullin Apr 06 '21 at 10:28
  • So is it still not guaranteed that document triggers are called only once? It seems like a big issue that has not further been discussed.. – Nicolas Degen Jul 02 '21 at 07:28
  • I am using the same to calculate the number of users currently in a room but because of 1 write per second, this sometimes fails. Any solution for that except the extensions? – Siddharth jha Aug 06 '21 at 07:04
  • @NicolasDegen there is a solution to the problem where triggers can be executed multiple times. It's described in the following article under "2.4 Incremental debounced" section: https://medium.com/firebase-tips-tricks/how-to-count-documents-in-firestore-a0527f792d04 – vir us Jan 13 '22 at 19:51
  • @Siddharthjha you can use a distributed counter to overcome "1 sec write" limitation. – vir us Jan 13 '22 at 19:53
  • Still ended up being helpful for me, specifically for the `FieldValue.increment(-1)` part! – ConcernedHobbit Jun 06 '22 at 19:38
  • import { increment } from 'firebase/firestore' – jessmzn Jun 16 '22 at 18:48
  • I am using the new aggregator "count()" function, but, for my collection, it's returning a count of 18 documents, when I know I have 140. What gives?! Is it because most of the documents are just ids, with no data fields? – Miss Henesy Apr 11 '23 at 13:44
41

Simplest way to do so is to read the size of a "querySnapshot".

db.collection("cities").get().then(function(querySnapshot) {      
    console.log(querySnapshot.size); 
});

You can also read the length of the docs array inside "querySnapshot".

querySnapshot.docs.length;

Or if a "querySnapshot" is empty by reading the empty value, which will return a boolean value.

querySnapshot.empty;
Ompel
  • 703
  • 4
  • 10
  • 130
    Be aware that each document "costs" one read. So if you count 100 items in a collection this way, you are being charged for 100 reads! – The Lone Coder Jan 13 '18 at 12:37
  • Correct, but there isn't any other way to sum up the number of documents in a collection. And if you already fetched the collection, reading the "docs" array won't require any more fetching, hence won't "cost" more readings. – Ompel Jan 14 '18 at 20:28
  • 9
    This reads all the documents in memory! Good luck with that for large datasets... – Dan Dascalescu Feb 23 '18 at 12:51
  • 215
    this is really unbelievable that Firebase Firestore don't have kind of `db.collection.count()`. Thinking dropping them only for this – Blue Bot May 10 '18 at 20:02
  • 25
    Especially for large collections, it is not fair to charge us as if we actually downloaded and used all documents. Count for a table (collection) is such a basic feature. Considering their pricing model and that Firestore was launched in 2017, it is just incredible that Google doesn't provide an alternative way to get the size of a collection. Until they don't implement it, they should at least avoid charging for it. – nibbana Apr 18 '19 at 13:46
  • this is just sneaky tactic by google to convert people to paid plan. its just cheap because if an application scales and gets users then anyways the application will be converted to a better paid plan. so throttling database functionality for developers is just malicious behaviour and developers should really question the importance of using such a service. – srinivas May 30 '20 at 07:57
  • @BlueBot it's still not possible? – Leonardo Rick Apr 25 '21 at 01:13
  • @LeonardoRick not sure, I stopped using Firebase since then as a DB. I use it for some of google services but thats it. no need in listing. So in short I don't know – Blue Bot Apr 25 '21 at 10:36
  • 1
    @BlueBot it does: https://firebase.google.com/docs/firestore/query-data/aggregation-queries – Jayen Oct 20 '22 at 08:20
34

As far as I know there is no build-in solution for this and it is only possible in the node sdk right now. If you have a

db.collection('someCollection')

you can use

.select([fields])

to define which field you want to select. If you do an empty select() you will just get an array of document references.

example:

db.collection('someCollection').select().get().then( (snapshot) => console.log(snapshot.docs.length) );

This solution is only a optimization for the worst case of downloading all documents and does not scale on large collections!

Also have a look at this:
How to get a count of number of documents in a collection with Cloud Firestore

Ashish
  • 6,791
  • 3
  • 26
  • 48
jbb
  • 530
  • 1
  • 5
  • 17
32

Aggregate count query just landed as a preview in Firestore.

Announced at the 2022 Firebase Summit: https://firebase.blog/posts/2022/10/whats-new-at-Firebase-Sumit-2022

Excerpt:

[Developer Preview] Count() function: With the new count function in Firstore [sic], you can now get the count of the matching documents when you run a query or read from a collection, without loading the actual documents, which saves you a lot of time.

Code sample they showed at the summit:

enter image description here

During the Q&A, someone asked about pricing for aggregated queries, and the answer the Firebase team provided was that it'll cost 1 / 1000th of the price of a read (rounded up to the nearest read, see comments below for more details), but will count all records that are part of the aggregate.

Johnny Oshika
  • 54,741
  • 40
  • 181
  • 275
21

Be careful counting number of documents for large collections. It is a little bit complex with firestore database if you want to have a precalculated counter for every collection.

Code like this doesn't work in this case:

export const customerCounterListener = 
    functions.firestore.document('customers/{customerId}')
    .onWrite((change, context) => {

    // on create
    if (!change.before.exists && change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count + 1
                     }))
    // on delete
    } else if (change.before.exists && !change.after.exists) {
        return firestore
                 .collection('metadatas')
                 .doc('customers')
                 .get()
                 .then(docSnap =>
                     docSnap.ref.set({
                         count: docSnap.data().count - 1
                     }))
    }

    return null;
});

The reason is because every cloud firestore trigger has to be idempotent, as firestore documentation say: https://firebase.google.com/docs/functions/firestore-events#limitations_and_guarantees

Solution

So, in order to prevent multiple executions of your code, you need to manage with events and transactions. This is my particular way to handle large collection counters:

const executeOnce = (change, context, task) => {
    const eventRef = firestore.collection('events').doc(context.eventId);

    return firestore.runTransaction(t =>
        t
         .get(eventRef)
         .then(docSnap => (docSnap.exists ? null : task(t)))
         .then(() => t.set(eventRef, { processed: true }))
    );
};

const documentCounter = collectionName => (change, context) =>
    executeOnce(change, context, t => {
        // on create
        if (!change.before.exists && change.after.exists) {
            return t
                    .get(firestore.collection('metadatas')
                    .doc(collectionName))
                    .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: ((docSnap.data() && docSnap.data().count) || 0) + 1
                        }));
        // on delete
        } else if (change.before.exists && !change.after.exists) {
            return t
                     .get(firestore.collection('metadatas')
                     .doc(collectionName))
                     .then(docSnap =>
                        t.set(docSnap.ref, {
                            count: docSnap.data().count - 1
                        }));
        }

        return null;
    });

Use cases here:

/**
 * Count documents in articles collection.
 */
exports.articlesCounter = functions.firestore
    .document('articles/{id}')
    .onWrite(documentCounter('articles'));

/**
 * Count documents in customers collection.
 */
exports.customersCounter = functions.firestore
    .document('customers/{id}')
    .onWrite(documentCounter('customers'));

As you can see, the key to prevent multiple execution is the property called eventId in the context object. If the function has been handled many times for the same event, the event id will be the same in all cases. Unfortunately, you must have "events" collection in your database.

Ferran Verdés
  • 306
  • 2
  • 5
  • 2
    They are phrasing this as if this behavior would be fixed in the 1.0 release. Amazon AWS functions suffer from the same problem. Something so simple as counting fields becomes complex, and expensive. – Marcelo Glasberg Oct 23 '18 at 15:28
  • Going to try this now since it seems like a better solution. Do you go back and purge your events collection ever? I was thinking of just adding a date field and purging older than a day or something just to keep the data set small (possibly 1mil+ events/day). Unless there's an easy way in FS to do that...only been using FS a few months. – Tym Pollack Dec 29 '18 at 20:11
  • 1
    Can we verify that `context.eventId` will always be the same on multiple invocations of the same trigger? In my testing it appears to be consistent, but I am unable to find any "official" documentation stating this. – Mike McLin Jan 04 '19 at 14:13
  • 2
    So after using this a while, I've found that, while this solution does work with exactly one write, which is great, if too many triggers fire from multiple docs being written at once and trying to update the same count doc, you can get contention errors from firestore. Have you encountered those, and how did you get around it? (Error: 10 ABORTED: Too much contention on these documents. Please try again.) – Tym Pollack Jan 06 '19 at 18:37
  • 1
    @TymPollack look at [distributed counters](https://firebase.google.com/docs/firestore/solutions/counters) document writes are limited to approx one update per second – Jamie Apr 05 '19 at 20:01
18

In 2020 this is still not available in the Firebase SDK however it is available in Firebase Extensions (Beta) however it's pretty complex to setup and use...

A reasonable approach

Helpers... (create/delete seems redundant but is cheaper than onUpdate)

export const onCreateCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(1);
  await statsDoc.set(countDoc, { merge: true });
};

export const onDeleteCounter = () => async (
  change,
  context
) => {
  const collectionPath = change.ref.parent.path;
  const statsDoc = db.doc("counters/" + collectionPath);
  const countDoc = {};
  countDoc["count"] = admin.firestore.FieldValue.increment(-1);
  await statsDoc.set(countDoc, { merge: true });
};

export interface CounterPath {
  watch: string;
  name: string;
}

Exported Firestore hooks


export const Counters: CounterPath[] = [
  {
    name: "count_buildings",
    watch: "buildings/{id2}"
  },
  {
    name: "count_buildings_subcollections",
    watch: "buildings/{id2}/{id3}/{id4}"
  }
];


Counters.forEach(item => {
  exports[item.name + '_create'] = functions.firestore
    .document(item.watch)
    .onCreate(onCreateCounter());

  exports[item.name + '_delete'] = functions.firestore
    .document(item.watch)
    .onDelete(onDeleteCounter());
});

In action

The building root collection and all sub collections will be tracked.

enter image description here

Here under the /counters/ root path

enter image description here

Now collection counts will update automatically and eventually! If you need a count, just use the collection path and prefix it with counters.

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const collectionCount = await db
  .doc('counters/' + collectionPath)
  .get()
  .then(snap => snap.get('count'));

Limitations

As this approach uses a single database and document, it is limited to the Firestore constraint of 1 Update per Second for each counter. It will be eventually consistent, but in cases where large amounts of documents are added/removed the counter will lag behind the actual collection count.

Ben Winding
  • 10,208
  • 4
  • 80
  • 67
  • 1
    Isn't this subjected to the same limitation "1 document update per second" ? – Ayyappa May 27 '20 at 17:32
  • Yes, but it is eventually consistent, meaning the collection count will _eventually_ align with the actual collection count, it's the easiest solution to implement and in many cases a brief lag in count is acceptable. – Ben Winding May 31 '20 at 12:46
  • Limitations: 10,000 per second (as per official documentation: https://firebase.google.com/products/extensions/firestore-counter) – Pooja Aug 25 '20 at 16:26
  • 3
    @Pooja that limitation is wrong, as it refers to _distributed_ counters, the above solution is _not_ distributed. – Ben Winding Aug 25 '20 at 23:54
  • WARNING - it's not really consistent meaning the count WILL deviate over time simply because firebase triggers can be executed multiple times - there is no guarantee it will be executed only once. The below article describes the problem and a possible solution in more detail: https://medium.com/firebase-tips-tricks/how-to-count-documents-in-firestore-a0527f792d04 – vir us Jan 13 '22 at 19:54
  • Thanks for the warning, I wasn't aware they could be executed more than once. I wondering if they could be converted to idempotent functions? https://cloud.google.com/blog/products/serverless/cloud-functions-pro-tips-building-idempotent-functions – Ben Winding Jan 15 '22 at 00:02
11

I agree with @Matthew, it will cost a lot if you perform such query.

[ADVICE FOR DEVELOPERS BEFORE STARTING THEIR PROJECTS]

Since we have foreseen this situation at the beginning, we can actually make a collection namely counters with a document to store all the counters in a field with type number.

For example:

For each CRUD operation on the collection, update the counter document:

  1. When you create a new collection/subcollection: (+1 in the counter) [1 write operation]
  2. When you delete a collection/subcollection: (-1 in the counter) [1 write operation]
  3. When you update an existing collection/subcollection, do nothing on the counter document: (0)
  4. When you read an existing collection/subcollection, do nothing on the counter document: (0)

Next time, when you want to get the number of collection, you just need to query/point to the document field. [1 read operation]

In addition, you can store the collections name in an array, but this will be tricky, the condition of array in firebase is shown as below:

// we send this
['a', 'b', 'c', 'd', 'e']
// Firebase stores this
{0: 'a', 1: 'b', 2: 'c', 3: 'd', 4: 'e'}

// since the keys are numeric and sequential,
// if we query the data, we get this
['a', 'b', 'c', 'd', 'e']

// however, if we then delete a, b, and d,
// they are no longer mostly sequential, so
// we do not get back an array
{2: 'c', 4: 'e'}

So, if you are not going to delete the collection , you can actually use array to store list of collections name instead of querying all the collection every time.

Hope it helps!

Angus
  • 3,680
  • 1
  • 12
  • 27
  • 2
    For a small collection, maybe. But keep in mind the Firestore document size limit is ~1MB, which, if the document IDs in a collection are auto-generated (20 bytes), then you'll only be able to store ~52,425 of them before the doc holding the array is too big. I guess as a workaround to that you could make a new doc every 50,000 elements, but then maintaining those arrays would be entirely unmanageable. Further, as the doc size grows, it will take longer to read and update, which will eventually make any other operations on it time out being in contention. – Tym Pollack Jan 10 '19 at 13:14
10

As of October 2022, Firestore has introduced a count() method on the client SDKs. Now you can count for a query without downloads.

For 1000 documents, it will charge you for 1 document read.

Web (v9)

Introduced in Firebase 9.11.0:

const collectionRef = collection(db, "cities");
const snapshot = await getCountFromServer(collectionRef);
console.log('count: ', snapshot.data().count);

Web V8

Not Available.

Node (Admin)

const collectionRef = db.collection('cities');
const snapshot = await collectionRef.count().get();
console.log(snapshot.data().count);

Android (Kotlin)

Introduced in firestore v24.4.0 (BoM 31.0.0):

val query = db.collection("cities")
val countQuery = query.count()
countQuery.get(AggregateSource.SERVER).addOnCompleteListener { task ->
    if (task.isSuccessful) {
        val snapshot = task.result
        Log.d(TAG, "Count: ${snapshot.count}")
    } else {
        Log.d(TAG, "Count failed: ", task.getException())
    }
}

Apple Platforms (Swift)

Introduced in Firestore v10.0.0:

do {
  let query = db.collection("cities")
  let countQuery = query.countAggregateQuery
  let snapshot = try await countQuery.aggregation(source: AggregateSource.server)
  print(snapshot.count)
} catch {
  print(error)
}
Mises
  • 4,251
  • 2
  • 19
  • 32
7

Increment a counter using admin.firestore.FieldValue.increment:

exports.onInstanceCreate = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onCreate((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(1),
    })
  );

exports.onInstanceDelete = functions.firestore.document('projects/{projectId}/instances/{instanceId}')
  .onDelete((snap, context) =>
    db.collection('projects').doc(context.params.projectId).update({
      instanceCount: admin.firestore.FieldValue.increment(-1),
    })
  );

In this example we increment an instanceCount field in the project each time a document is added to the instances sub collection. If the field doesn't exist yet it will be created and incremented to 1.

The incrementation is transactional internally but you should use a distributed counter if you need to increment more frequently than every 1 second.

It's often preferable to implement onCreate and onDelete rather than onWrite as you will call onWrite for updates which means you are spending more money on unnecessary function invocations (if you update the docs in your collection).

Dominic
  • 62,658
  • 20
  • 139
  • 163
6

No, there is no built-in support for aggregation queries right now. However there are a few things you could do.

The first is documented here. You can use transactions or cloud functions to maintain aggregate information:

This example shows how to use a function to keep track of the number of ratings in a subcollection, as well as the average rating.

exports.aggregateRatings = firestore
  .document('restaurants/{restId}/ratings/{ratingId}')
  .onWrite(event => {
    // Get value of the newly added rating
    var ratingVal = event.data.get('rating');

    // Get a reference to the restaurant
    var restRef = db.collection('restaurants').document(event.params.restId);

    // Update aggregations in a transaction
    return db.transaction(transaction => {
      return transaction.get(restRef).then(restDoc => {
        // Compute new number of ratings
        var newNumRatings = restDoc.data('numRatings') + 1;

        // Compute new average rating
        var oldRatingTotal = restDoc.data('avgRating') * restDoc.data('numRatings');
        var newAvgRating = (oldRatingTotal + ratingVal) / newNumRatings;

        // Update restaurant info
        return transaction.update(restRef, {
          avgRating: newAvgRating,
          numRatings: newNumRatings
        });
      });
    });
});

The solution that jbb mentioned is also useful if you only want to count documents infrequently. Make sure to use the select() statement to avoid downloading all of each document (that's a lot of bandwidth when you only need a count). select() is only available in the server SDKs for now so that solution won't work in a mobile app.

Sam Stern
  • 24,624
  • 13
  • 93
  • 124
  • 1
    This solution is not idempotent, so any triggers that fire more than once will throw off your number of ratings and average. – Tym Pollack Jan 10 '19 at 13:28
6

UPDATE 11/20

I created an npm package for easy access to a counter function: https://code.build/p/9DicAmrnRoK4uk62Hw1bEV/firestore-counters


I created a universal function using all these ideas to handle all counter situations (except queries).

The only exception would be when doing so many writes a second, it slows you down. An example would be likes on a trending post. It is overkill on a blog post, for example, and will cost you more. I suggest creating a separate function in that case using shards: https://firebase.google.com/docs/firestore/solutions/counters

// trigger collections
exports.myFunction = functions.firestore
    .document('{colId}/{docId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// trigger sub-collections
exports.mySubFunction = functions.firestore
    .document('{colId}/{docId}/{subColId}/{subDocId}')
    .onWrite(async (change: any, context: any) => {
        return runCounter(change, context);
    });

// add change the count
const runCounter = async function (change: any, context: any) {

    const col = context.params.colId;

    const eventsDoc = '_events';
    const countersDoc = '_counters';

    // ignore helper collections
    if (col.startsWith('_')) {
        return null;
    }
    // simplify event types
    const createDoc = change.after.exists && !change.before.exists;
    const updateDoc = change.before.exists && change.after.exists;

    if (updateDoc) {
        return null;
    }
    // check for sub collection
    const isSubCol = context.params.subDocId;

    const parentDoc = `${countersDoc}/${context.params.colId}`;
    const countDoc = isSubCol
        ? `${parentDoc}/${context.params.docId}/${context.params.subColId}`
        : `${parentDoc}`;

    // collection references
    const countRef = db.doc(countDoc);
    const countSnap = await countRef.get();

    // increment size if doc exists
    if (countSnap.exists) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.update(countRef, { count: i });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        const colRef = db.collection(change.after.ref.parent.path);
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(colRef);
            return t.set(countRef, { count: colSnap.size });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}

This handles events, increments, and transactions. The beauty in this, is that if you are not sure about the accuracy of a document (probably while still in beta), you can delete the counter to have it automatically add them up on the next trigger. Yes, this costs, so don't delete it otherwise.

Same kind of thing to get the count:

const collectionPath = 'buildings/138faicnjasjoa89/buildingContacts';
const colSnap = await db.doc('_counters/' + collectionPath).get();
const count = colSnap.get('count');

Also, you may want to create a cron job (scheduled function) to remove old events to save money on database storage. You need at least a blaze plan, and there may be some more configuration. You could run it every sunday at 11pm, for example. https://firebase.google.com/docs/functions/schedule-functions

This is untested, but should work with a few tweaks:

exports.scheduledFunctionCrontab = functions.pubsub.schedule('5 11 * * *')
    .timeZone('America/New_York')
    .onRun(async (context) => {

        // get yesterday
        const yesterday = new Date();
        yesterday.setDate(yesterday.getDate() - 1);

        const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
        const eventFilterSnap = await eventFilter.get();
        eventFilterSnap.forEach(async (doc: any) => {
            await doc.ref.delete();
        });
        return null;
    });

And last, don't forget to protect the collections in firestore.rules:

match /_counters/{document} {
  allow read;
  allow write: if false;
}
match /_events/{document} {
  allow read, write: if false;
}

Update: Queries

Adding to my other answer if you want to automate query counts as well, you can use this modified code in your cloud function:

    if (col === 'posts') {

        // counter reference - user doc ref
        const userRef = after ? after.userDoc : before.userDoc;
        // query reference
        const postsQuery = db.collection('posts').where('userDoc', "==", userRef);
        // add the count - postsCount on userDoc
        await addCount(change, context, postsQuery, userRef, 'postsCount');

    }
    return delEvents();

Which will automatically update the postsCount in the userDocument. You could easily add other one to many counts this way. This just gives you ideas of how you can automate things. I also gave you another way to delete the events. You have to read each date to delete it, so it won't really save you to delete them later, just makes the function slower.

/**
 * Adds a counter to a doc
 * @param change - change ref
 * @param context - context ref
 * @param queryRef - the query ref to count
 * @param countRef - the counter document ref
 * @param countName - the name of the counter on the counter document
 */
const addCount = async function (change: any, context: any, 
  queryRef: any, countRef: any, countName: string) {

    // events collection
    const eventsDoc = '_events';

    // simplify event type
    const createDoc = change.after.exists && !change.before.exists;

    // doc references
    const countSnap = await countRef.get();

    // increment size if field exists
    if (countSnap.get(countName)) {
        // createDoc or deleteDoc
        const n = createDoc ? 1 : -1;
        const i = admin.firestore.FieldValue.increment(n);

        // create event for accurate increment
        const eventRef = db.doc(`${eventsDoc}/${context.eventId}`);

        return db.runTransaction(async (t: any): Promise<any> => {
            const eventSnap = await t.get(eventRef);
            // do nothing if event exists
            if (eventSnap.exists) {
                return null;
            }
            // add event and update size
            await t.set(countRef, { [countName]: i }, { merge: true });
            return t.set(eventRef, {
                completed: admin.firestore.FieldValue.serverTimestamp()
            });
        }).catch((e: any) => {
            console.log(e);
        });
        // otherwise count all docs in the collection and add size
    } else {
        return db.runTransaction(async (t: any): Promise<any> => {
            // update size
            const colSnap = await t.get(queryRef);
            return t.set(countRef, { [countName]: colSnap.size }, { merge: true });
        }).catch((e: any) => {
            console.log(e);
        });;
    }
}
/**
 * Deletes events over a day old
 */
const delEvents = async function () {

    // get yesterday
    const yesterday = new Date();
    yesterday.setDate(yesterday.getDate() - 1);

    const eventFilter = db.collection('_events').where('completed', '<=', yesterday);
    const eventFilterSnap = await eventFilter.get();
    eventFilterSnap.forEach(async (doc: any) => {
        await doc.ref.delete();
    });
    return null;
}

I should also warn you that universal functions will run on every onWrite call period. It may be cheaper to only run the function on onCreate and on onDelete instances of your specific collections. Like the noSQL database we are using, repeated code and data can save you money.

Jonathan
  • 3,893
  • 5
  • 46
  • 77
5

There is no direct option available. You cant't do db.collection("CollectionName").count(). Below are the two ways by which you can find the count of number of documents within a collection.

1 :- Get all the documents in the collection and then get it's size.(Not the best Solution)

db.collection("CollectionName").get().subscribe(doc=>{
console.log(doc.size)
})

By using above code your document reads will be equal to the size of documents within a collection and that is the reason why one must avoid using above solution.

2:- Create a separate document with in your collection which will store the count of number of documents in the collection.(Best Solution)

db.collection("CollectionName").doc("counts")get().subscribe(doc=>{
console.log(doc.count)
})

Above we created a document with name counts to store all the count information.You can update the count document in the following way:-

  • Create a firestore triggers on the document counts
  • Increment the count property of counts document when a new document is created.
  • Decrement the count property of counts document when a document is deleted.

w.r.t price (Document Read = 1) and fast data retrieval the above solution is good.

Nipun Madan
  • 169
  • 1
  • 10
4

A workaround is to:

write a counter in a firebase doc, which you increment within a transaction everytime you create a new entry

You store the count in a field of your new entry (i.e: position: 4).

Then you create an index on that field (position DESC).

You can do a skip+limit with a query.Where("position", "<" x).OrderBy("position", DESC)

Hope this helps!

Fatih Acet
  • 28,690
  • 9
  • 51
  • 58
Kathan Shah
  • 1,655
  • 17
  • 25
3

I have try a lot with different approaches. And finally, I improve one of the methods. First you need to create a separate collection and save there all events. Second you need to create a new lambda to be triggered by time. This lambda will Count events in event collection and clear event documents. Code details in article. https://medium.com/@ihor.malaniuk/how-to-count-documents-in-google-cloud-firestore-b0e65863aeca

Ihor Malaniuk
  • 151
  • 1
  • 8
  • 3
    Please include the relevant details and code [in the answer itself](https://meta.stackexchange.com/questions/8231/are-answers-that-just-contain-links-elsewhere-really-good-answers), pointing people at your blog posts isn't really the point of StackOverflow. – DBS May 27 '20 at 16:04
2

one of the fast + money saver trick is that:-

make a doc and store a 'count' variable in firestore, when user add new doc in the collection, increase that variable, and when user delete a doc, decrease variable. e.g. updateDoc(doc(db, "Count_collection", "Count_Doc"), {count: increment(1)});

note: use (-1) for decreasing, (1) for increasing count

How it save money and time:-

  1. you(firebase) don't need to loop through the collection, nor browser needs to load whole collection to count number of docs.
  2. all the counts are save in a doc of only one variable named "count" or whatever, so less than 1kb data is used, and it use only 1 reads in firebase firestore.
1

Solution using pagination with offset & limit:

public int collectionCount(String collection) {
        Integer page = 0;
        List<QueryDocumentSnapshot> snaps = new ArrayList<>();
        findDocsByPage(collection, page, snaps);
        return snaps.size();
    }

public void findDocsByPage(String collection, Integer page, 
                           List<QueryDocumentSnapshot> snaps) {
    try {
        Integer limit = 26000;
        FieldPath[] selectedFields = new FieldPath[] { FieldPath.of("id") };
        List<QueryDocumentSnapshot> snapshotPage;
        snapshotPage = fireStore()
                        .collection(collection)
                        .select(selectedFields)
                        .offset(page * limit)
                        .limit(limit)
                        .get().get().getDocuments();    
        if (snapshotPage.size() > 0) {
            snaps.addAll(snapshotPage);
            page++;
            findDocsByPage(collection, page, snaps);
        }
    } catch (InterruptedException | ExecutionException e) {
        e.printStackTrace();
    }
}
  • findDocsPage it's a recursive method to find all pages of collection

  • selectedFields for otimize query and get only id field instead full body of document

  • limit max size of each query page

  • page define inicial page for pagination

From the tests I did it worked well for collections with up to approximately 120k records!

ℛɑƒæĿᴿᴹᴿ
  • 4,983
  • 4
  • 38
  • 58
  • 2
    Keep in mind with the backend offset function, you are getting charged reads for all documents that come before the offset document... so ```offset(119223)``` would charge you for 119,223 reads, which can get VERY expensive if used all the time. If you know the document to ```startAt(doc)```, that can help, but usually you don't have that information or you wouldn't be searching! – Jonathan Nov 18 '20 at 00:59
1

Firestore is introducing a new Query.count() that fetches the count of a query without fetching the docs.

This would allow to simply query all collection items and get the count of that query.

Ref:

Nicolas Degen
  • 1,522
  • 2
  • 15
  • 24
1

There's a new build in function since version 9.11.0 called getCountFromServer(), which fetches the number of documents in the result set without actually downloading the documents.

https://firebase.google.com/docs/reference/js/firestore_#getcountfromserver

hovado
  • 4,474
  • 1
  • 24
  • 30
0

Took me a while to get this working based on some of the answers above, so I thought I'd share it for others to use. I hope it's useful.

'use strict';

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();
const db = admin.firestore();

exports.countDocumentsChange = functions.firestore.document('library/{categoryId}/documents/{documentId}').onWrite((change, context) => {

    const categoryId = context.params.categoryId;
    const categoryRef = db.collection('library').doc(categoryId)
    let FieldValue = require('firebase-admin').firestore.FieldValue;

    if (!change.before.exists) {

        // new document created : add one to count
        categoryRef.update({numberOfDocs: FieldValue.increment(1)});
        console.log("%s numberOfDocs incremented by 1", categoryId);

    } else if (change.before.exists && change.after.exists) {

        // updating existing document : Do nothing

    } else if (!change.after.exists) {

        // deleting document : subtract one from count
        categoryRef.update({numberOfDocs: FieldValue.increment(-1)});
        console.log("%s numberOfDocs decremented by 1", categoryId);

    }

    return 0;
});
Rob Phillips
  • 295
  • 3
  • 9
0

This uses counting to create numeric unique ID. In my use, I will not be decrementing ever, even when the document that the ID is needed for is deleted.

Upon a collection creation that needs unique numeric value

  1. Designate a collection appData with one document, set with .doc id only
  2. Set uniqueNumericIDAmount to 0 in the firebase firestore console
  3. Use doc.data().uniqueNumericIDAmount + 1 as the unique numeric id
  4. Update appData collection uniqueNumericIDAmount with firebase.firestore.FieldValue.increment(1)
firebase
    .firestore()
    .collection("appData")
    .doc("only")
    .get()
    .then(doc => {
        var foo = doc.data();
        foo.id = doc.id;

        // your collection that needs a unique ID
        firebase
            .firestore()
            .collection("uniqueNumericIDs")
            .doc(user.uid)// user id in my case
            .set({// I use this in login, so this document doesn't
                  // exist yet, otherwise use update instead of set
                phone: this.state.phone,// whatever else you need
                uniqueNumericID: foo.uniqueNumericIDAmount + 1
            })
            .then(() => {

                // upon success of new ID, increment uniqueNumericIDAmount
                firebase
                    .firestore()
                    .collection("appData")
                    .doc("only")
                    .update({
                        uniqueNumericIDAmount: firebase.firestore.FieldValue.increment(
                            1
                        )
                    })
                    .catch(err => {
                        console.log(err);
                    });
            })
            .catch(err => {
                console.log(err);
            });
    });
0
var variable=0
variable=variable+querySnapshot.count

then if you are to use it on a String variable then

let stringVariable= String(variable)
Natalia Kolisnyk
  • 287
  • 2
  • 10
0

This feature is now supported in FireStore, albeit in Beta. Here are the official Firebase docs

0

With the new version of Firebase, you can now run aggregated queries! Simply write

.count().get(); 

after your query.

Suraj Rao
  • 29,388
  • 11
  • 94
  • 103
mik
  • 799
  • 9
  • 31
0

As it stands, firebase only allows server-side count, like this

const collectionRef = db.collection('cities');
const snapshot = await collectionRef.count().get();
console.log(snapshot.data().count);

Please not this is for nodeJS

Manbus
  • 706
  • 4
  • 9
0

New feature available in Firebase/Firestore provides a count of documents in a collection:

See this thread to see how to achieve it, with an example.

How To Count Number of Documents in a Collection in Firebase Firestore With a WHERE query in react.js

Michael Cockinos
  • 165
  • 2
  • 15
0

According to this documentation Cloud Firestore supports the count() aggregation query and is available in preview.

The Flutter/Dart code was missing (at the time of writing this) so I played around with it and the following function seems to work:

  Future<int> getCount(String path) async {
    var collection = _fireStore.collection(path);
    var countQuery = collection.count();
    var snapShot = await countQuery.get(source: AggregateSource.server);
    return snapShot.count;
  }
Niels
  • 1,366
  • 15
  • 21
0

Here is the way you can count the documentIds and get the complete list:

app.post('/read-whole-collection', async (req, res) => {
    fireStore.initializeApp({
        credential: fireStore.credential.cert(FS_OBJ)
    });

    const db = fireStore.firestore();
    const { fromTable, isSize } = req.body;
    try {
        const adminRef = await db.collection(fromTable).listDocuments();
        const size = adminRef.length;
        console.log('size', size);
        if(isSize) {
            res.send({size, fromTable});
        }

        const data = adminRef.map((doc, index) => {
            return doc.id;
        });
        res.send({size, length: data.length, data });
    } catch (error) {
        console.log({ message: 'read-whole-collection : Something went wrong in reading data', error: error });
        res.send({ message: 'read-whole-collection: Something went wrong in reading data', error: error });
    };
});

Hit the URL from postman:

curl --location --request POST 'http://localhost:3000/read-whole-collection' \
--header 'Content-Type: application/json' \
--data-raw '{
    "fromTable": "Users",
    "isSize": false
}'
Shubham Verma
  • 8,783
  • 6
  • 58
  • 79
-1
firebaseFirestore.collection("...").addSnapshotListener(new EventListener<QuerySnapshot>() {
        @Override
        public void onEvent(QuerySnapshot documentSnapshots, FirebaseFirestoreException e) {

            int Counter = documentSnapshots.size();

        }
    });
-1

Along with my npm package adv-firestore-functions above, you can also just use firestore rules to force a good counter:

Firestore Rules

function counter() {
  let docPath = /databases/$(database)/documents/_counters/$(request.path[3]);
  let afterCount = getAfter(docPath).data.count;
  let beforeCount = get(docPath).data.count;
  let addCount = afterCount == beforeCount + 1;
  let subCount = afterCount == beforeCount - 1;
  let newId = getAfter(docPath).data.docId == request.path[4];
  let deleteDoc = request.method == 'delete';
  let createDoc = request.method == 'create';
  return (newId && subCount && deleteDoc) || (newId && addCount && createDoc);
}

function counterDoc() {
  let doc = request.path[4];
  let docId = request.resource.data.docId;
  let afterCount = request.resource.data.count;
  let beforeCount = resource.data.count;
  let docPath = /databases/$(database)/documents/$(doc)/$(docId);
  let createIdDoc = existsAfter(docPath) && !exists(docPath);
  let deleteIdDoc = !existsAfter(docPath) && exists(docPath);
  let addCount = afterCount == beforeCount + 1;
  let subCount = afterCount == beforeCount - 1;
  return (createIdDoc && addCount) || (deleteIdDoc && subCount);
}

and use them like so:

match /posts/{document} {
  allow read;
  allow update;
  allow create: if counter();
  allow delete: if counter();
}
match /_counters/{document} {
  allow read;
  allow write: if counterDoc();
}

Frontend

UPDATE: 2/26/23 - I added these functions to j-firebase package.

Replace your set and delete functions with these:

set

async setDocWithCounter(
  ref: DocumentReference<DocumentData>,
  data: {
    [x: string]: any;
  },
  options: SetOptions): Promise<void> {

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const refSnap = await getDoc(ref);

  // don't increase count if edit
  if (refSnap.exists()) {
    await setDoc(ref, data, options);

    // increase count
  } else {
    const batch = writeBatch(this.afs);
    batch.set(ref, data, options);

    // if count exists
    if (countSnap.exists()) {
      batch.update(countRef, {
        count: increment(1),
        docId: ref.id
      });
      // create count
    } else {
      // will only run once, should not use
      // for mature apps
      const colRef = collection(this.afs, col);
      const colSnap = await getDocs(colRef);
      batch.set(countRef, {
        count: colSnap.size + 1,
        docId: ref.id
      });
    }
    batch.commit();
  }
}

delete

async delWithCounter(
  ref: DocumentReference<DocumentData>
): Promise<void> {

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const batch = writeBatch(this.afs);

  // if count exists
  batch.delete(ref);
  if (countSnap.exists()) {
    batch.update(countRef, {
      count: increment(-1),
      docId: ref.id
    });
  }
  /*
  if ((countSnap.data() as any).count == 1) {
    batch.delete(countRef);
  }*/
  batch.commit();
}

see here for more info...

J

Jonathan
  • 3,893
  • 5
  • 46
  • 77
-9

So my solution for this problem is a bit non-technical, not super precise, but good enough for me.

enter image description here

Those are my documents. As I have a lot of them (100k+) there are 'laws of big numbers' happening. I can assume that there is less-or-more equal number of items having id starting with 0, 1, 2, etc.

So what I do is I scroll my list till I get into id's starting with 1, or with 01, depending on how long you have to scroll

enter image description here

here we are.

Now, having scrolled so far, I open the inspector and see how much did I scroll and divide it by height of single element

enter image description here

Had to scroll 82000px to get items with id starting with 1. Height of single element is 32px.

It means I have 2500 with id starting with 0, so now I multiply it by number of possible 'starting char'. In firebase it can be A-Z, a-z, 0-9 which means it's 24 + 24 + 10 = 58.

It means I have ~~2500*58 so it gives roughly 145000 items in my collection.

Summarizing: What is wrong with you firebase?

Adam Pietrasiak
  • 12,773
  • 9
  • 78
  • 91
  • 2
    Well, I just need to count it from time to time to have any idea about the growth of my app data. TBH I think it's not my idea that is ridiculous, but lack of a simple 'count' feature in firebase. This is good enough for me and other answers here seem like annoying overkill. Single `measurement takes me ~3minutes which is probably way faster than setting up other solutions listed here. – Adam Pietrasiak May 05 '21 at 19:20