0

I am using Google Cloud Functions and read, write operations are performed on the Firestore thru these Cloud Functions. We are seeing unexpectedly high number of read operations on Firestore, the source of which I am unable to figure out.

Not more than 20K documents are generated on a daily basis. But the daily read count is usually more than 25,000,000

What I am looking for, is ways to identify the root cause of these high number of reads in the Cloud Functions.

To start with, I have captured the size of the results of all the Firestore get() methods in Cloud Functions. But the sum total of all the sizes is much much much lower than the read count I mentioned above.

Need suggestions on ways/practices to identify the source from where these high reads are generating.

Rajesh Sharma
  • 29
  • 1
  • 4
  • 2
    A common cause of this is keeping the Firestore console open during development. To show the documents, the console needs to read them and those count as charged document reads. -- If that does not explain the count, please edit your question to show the [minimal code that reproduces the problem](http://stackoverflow.com/help/mcve). – Frank van Puffelen Dec 12 '21 at 17:12
  • Unfortunately, have nothing that can reproduce the problem. That's where the struggle is. – Rajesh Sharma Dec 13 '21 at 14:34

1 Answers1

0

You can use a SnapshotListener as a workaround, which allows us to listen for changes in real-time.

You will be charged for readings as if we had sent a new query if the listener is disconnected for more than 30 minutes. If the listener is disconnected every 31 minutes in the worst-case scenario, we will be charged 50 reads each time.

As a result, this technique is only practicable when the listener is not frequently disconnected.

According to the documentation, I found you can reduce the number of reads using get(). In each document in the collection, you must add a new property named lastModified of type Date. When you create a new document or edit an existing one, you must use FieldValue.serverTimestamp() to set or update the field's value.

Reviewing Firebase documentation, I found that high read or write rates to lexicographically close documents need to be avoided to avoid contention faults in your application. Hotspotting is the term for this problem, and it can occur if your program does any of the following:

  • Creates new documents at a rapid rate and assigns its own IDs that are monotonically rising.
  • A scatter algorithm is used by Cloud Firestore to assign document IDs.
  • If you use automated document IDs to create new documents, you must not see hotspotting on writes.
  • In a collection with few documents, creates new documents at a fast rate.
  • Creates new documents at a rapid rate with a monotonically growing field, such as a timestamp.
  • Deletes a large number of documents from a collection.
  • Writes to the database at a rapid rate without growing traffic gradually.
Jose German Perez Sanchez
  • 1,103
  • 4
  • 16