10

Before creating a new app I wanna make sure I get the pricing model correct.

For example in a phonebook app, I have a collection called userList that has a list of users which are individual documents.

I have 50k users on my list, which means I have 50k documents in my collection.

If I were to get the userList collection it will read all 50k documents.

FireStore allows 50k document reads. Does that mean 50k document reads in total or 50k document read per document?

As in the example of my phonebook app if it is 50k document reads in total I will run out of the free limit in just one get call.

James Z
  • 12,209
  • 10
  • 24
  • 44
krv
  • 2,830
  • 7
  • 40
  • 79

3 Answers3

15

If you actually have to pull an entire collection of 50k documents, the question you likely should be asking is how to properly structure a Firestore Database.

More than likely you need to filter these documents based on some criteria within them by using the query WHERE clause. Having each client device hold 50k documents locally sounds like poor database planning and possibly a security risk.

Each returned document from your query counts as 1 read. If there are no matches to your query, 1 read is charged. If there are 50k matches, there are 50k reads charged.

For example, you can retrieve the logged in user's document and be charged 1 read with something like:

db.collection('userList').where('uid', '==', clientUID)

Note: As of 10/2018 Firestore charges 6 cents (USD) per 100k reads after the first 50k/ day.

Billal Begueradj
  • 20,717
  • 43
  • 112
  • 130
JavaBeast
  • 766
  • 3
  • 11
  • 28
  • But this was an extreme example to get my doubt clear. A common scenario would be a list of say 50 data items in doc. And say your app has 1000 daily users, in that case as well firestore would run out of free limits. Will be using real-time database as my data has no requirement of complex querying. – krv Oct 07 '18 at 03:54
  • 1
    Keep in mind the number of "data items" within a doc is irrelevant. When Firestore returns the doc from query it returns all items within it (bundled as an object) automatically, which counts as exactly 1 read. For 1000 users to each retrieve their entire user document, a total of 1000 reads would be charged. Based on my calculations of my own app (similar to SO), I anticipate a $200/ month bill @ 20,000 monthly users, but we'll see when (not if) I get there! – JavaBeast Oct 07 '18 at 04:13
  • 2
    "There is a minimum charge of one document read for each query that you perform, even if the query returns no results." https://firebase.google.com/docs/firestore/pricing – Hiwa Oct 31 '18 at 10:26
  • @Hiwa thanks for catching that! I have updated the answer appropriately. – JavaBeast Nov 02 '18 at 05:34
13

The free quota is for your entire project. So you're allowed 50.000 document reads under the entire project.

Reading 50K user profile documents will indeed use that free quota in one go.

Reading large numbers of documents is in general something you should try to prevent when using NoSQL databases.

The client apps that access Firestore should only read data that they're going to immediately show to the user. And there's no way you'll fit 50K users on a screen.

So more likely you have a case where you're aggregating over the user collection. E.g. things like:

  • Count the number of users
  • Count the number of users named Frank
  • Calculate the average length of the user names

NoSQL databases are usually more limited in their query capabilities than traditional relational databases, because they focus on ensuring read-scalability. You'll frequently do extra work when something is written to the database, if in exchange you can get better performance when reading from the database.

For better performance you'll want to store these aggregation values in the database, and then update them whenever a user profile is written. So you'll have a "userCount", a document with "userCount for each unique username", and a "averageUsernameLength".

For an example of how to run such aggregation queries, see: https://firebase.google.com/docs/firestore/solutions/aggregation. For lower write volumes, you can also consider using Cloud Functions to update the counters.

Frank van Puffelen
  • 565,676
  • 79
  • 828
  • 807
  • I agree with you. But this was an extreme example to get my doubt clear. A common scenario would be a list of say 50 data items in doc. And say your app has 1000 daily users, in that case as well firestore would run out of free limits. Will be using real-time database as my data has no requirement of complex querying. – krv Feb 17 '18 at 06:39
  • 1
    @Frank Van Puffelen, I have an additional question you may be able to answer. Lets say I have a news site and the home page has 10 articles. Every time a user goes to the home page, will it count 10 reads against my quota? Or will it count every item like 'title', 'body', 'author' as well to count 30 reads every time a user visits my site? – gbland777 Nov 17 '18 at 20:56
  • @gbland777 this will depend on both your site structure and your firestore structure. For my app each "item" of the news feed would have all its displayed data contained in 1 document each. This would indeed count as 10 reads to retrieve 10 news feed items then. I then cache this data and any time user clicks back to home it will display cached results, and charge 0 reads. Additionally you can use something like VuexFire (depending on tec used) to bind to a query which automatically updates when db updates. read more at firestore docs. – JavaBeast Nov 19 '18 at 18:59
  • 1
    Alternatively you can create a derived document that contains everything you show on a single page in the app/site. I'd recommend checking out Todd's [Get to know Cloud Firestore video series](https://www.youtube.com/watch?v=v_hR4K4auoQ&list=PLl-K7zZEsYLluG5MCVEzXAQ7ACZBCuZgZ), specifically episodes 3 and 5 for this. – Frank van Puffelen Nov 19 '18 at 21:44
2

Don't call all users in one go. You can limit your query to get a limited number of users. And when a user will scroll your query will get more users. And as no one is going to scroll fro 50k users so you can get rid of a bundle of cost. This is something like saving memory in case of recycle view.

Compile error end
  • 219
  • 4
  • 11